I hesitate to be too negative here, but I've got some pretty significant concerns about this one... Since it depends on a sleep call on the server side, I'm not really sure what it's intended to...
I hesitate to be too negative here, but I've got some pretty significant concerns about this one...
Since it depends on a sleep call on the server side, I'm not really sure what it's intended to gain over repeated short polling from the client? It'd be the same level of not-quite-realtime, but be radically more scalable since each client would block a PHP process for a few milliseconds every 5 seconds, rather than 300 full seconds at a time. Even assuming an excessively slow DB query and accounting for overhead, repeated polling from the client would allow for more than 50x the throughput on an otherwise identical server configuration.
The client keeps waiting for a response and this will inevitably block a thread on your web server.
This definitely isn't inevitable. Non-blocking asynchronous IO is pretty standard nowadays - nginx uses it, for example, and then farms those requests out to a fixed pool of PHP workers. In fact, using nginx itself to repeatedly poll those PHP processes while keeping the client connection open would make for an interesting experiment! I don't think it'd be the optimal choice, and I imagine some hacks would be involved to make it happen at all, but it'd be a significant improvement over blocking a PHP process for each user.
A notable exception here is node.js. Due to its uniquely architectured single-threaded event based model, this problem isn't faced.
Most major languages have at least one mainstream event loop implementation, including PHP itself (Revolt, Amp, or ReactPHP), and while we could probably quibble about minor differences and underlying details that make each one unique, the architecture and usage in all of them tends to be pretty similar to that of V8, which is Node's JS engine. They definitely all make it straightforward to break the thread-per-connection model if you want to.
The same principle applies even when you go for other solutions like websocket, cometd, etc. but these libraries or components hide this "wait" implementation away from your code.
Any implementation that I'm aware of will use a thread pool and non-blocking IO. I guess if you go right down to the OS level you can argue that something is doing the waiting, but any developer I know would consider that to be misleading at best - you can comfortably support literal millions of concurrent websockets on a single CPU core if they're all just idle.
Thanks for going through the post, Greg, I get the crux of what you're trying to say here. I'm trying to conserve network bandwidth here. If the polling happened every 5 seconds on client side,...
Thanks for going through the post, Greg, I get the crux of what you're trying to say here.
Since it depends on a sleep call on the server side, I'm not really sure what it's intended to gain over repeated short polling from the client?
I'm trying to conserve network bandwidth here. If the polling happened every 5 seconds on client side, there is a concern those concurrent requests might choke/waste the net bandwidth during active use, especially considering all those 30-40 users are in the same LAN/WLAN. The app is still under testing stage and, in fact, it's still being debated whether to host the PHP scripts on the internet cloud or on a local machine in the LAN itself. If that happens, bandwidth concerns may not arise but I think long-polling is still better and more efficient than short?
It'd be the same level of not-quite-realtime, but be radically more scalable since each client would block a PHP process for a few milliseconds every 5 seconds, rather than 300 full seconds at a time.
I don't think the process will be blocked for full 300 seconds. That's why I've included sleep() in the loop which "idles" the block for 5 seconds at each iteration. Effectively, these sleep chunks should release the CPU and make it idle again for other processes to use. At least in theory, it should scale to a high enough number of requests considering that all processes won't be active at once and most will be in idle state state due to these sleep() chunks?
Consider that rotating chair analogy where there are limited number of seats and a slightly higher number of users. But funnily, each user will be pulled up to the roof and stuck there for 5 seconds so that the chair is vacated during that time for other users to use!
An HTTP request like this should only send a few hundred bytes each way, so I really wouldn't worry about it for 40 users. A quick back of the envelope calculation suggests you'd be able to...
An HTTP request like this should only send a few hundred bytes each way, so I really wouldn't worry about it for 40 users. A quick back of the envelope calculation suggests you'd be able to support all of your users on a single 56Kbps dial up connection if you needed to, and if you do use any kind of public internet facing server you'll get more requests than this just from bots sweeping for known vulnerabilities, so I'm pretty confident in saying bandwidth is a non-issue for your use case.
You're right that it is still wasteful in the abstract, and it might eventually become an issue at significant scale, but that's the exact concern that websockets are designed to solve. As it is, if you need a quick and dirty solution without any dependencies, short polling will scale to a few thousand concurrent users on any reasonable modern connection.
In comparison, long polling is limited by your number of concurrent PHP processes. You're right that CPU isn't a major concern with sleep, but memory absolutely is: it's been years since I last used PHP in production, but from what I'm seeing in docs the memory overhead is still tens of MB. The normal way to handle this is for the web server to keep a limited pool of PHP workers, assign each one to a given incoming request for the minimum time possible, and then pass it along to the next one if there are more requests waiting than available processes. The sleep might not block the CPU, but it will block the PHP process and make it unavailable for reuse by the server, meaning that you need an entire process per user with all the overheads that entails, rather than just a few processes in total that can handle the short polling requests in sequence. Even being conservative with the RAM estimations, you're limited to tens or hundreds of users on a server that would otherwise have scaled to thousands.
There are two really important things in understanding the tradeoffs here:
Thread-per-connection or process-per-connection models are uncommon nowadays, and the architecture of most tools you might use here (websocket implementations, concurrency libraries, web servers, etc.) is likely to have much more in common with V8 than with the old Apache prefork MPM, even if it doesn't use the exact same concurrency model.
Connections (whether threaded or event driven) are much more lightweight than PHP processes, and if at all possible you want to use a few of the latter to serve many of the former. Keeping a PHP process in an idle loop prevents it from being reused for the next connection.
So, I’ve kind of nerd sniped myself here and tried to figure out how to solve this efficiently in pure PHP without “cheating” (by an entirely arbitrary definition that I just made up: no...
So, I’ve kind of nerd sniped myself here and tried to figure out how to solve this efficiently in pure PHP without “cheating” (by an entirely arbitrary definition that I just made up: no libraries, no C extensions, and no fibers).
It looks like the key is stream_select, which takes a timeout as an upper limit but immediately returns on stream activity if it occurs before that. It gives that lower level non-blocking behaviour that’s otherwise impossible in a single threaded language like PHP. Along with a stream_socket_server you can basically treat it like a network-interruptible sleep call, which in turn allows you to handle multiple connections per process - you can just add them to an array as they come in, even while you’re idling on the loop for an existing connection. Add a bit of calculation to keep track of how long is left on each connection’s 5 second timer each time the loop runs, set your timeout value to the shortest one, and you’ve just built yourself a rudimentary event loop!
I wouldn’t call it a good idea: it’s way overcomplicated compared to just repeatedly polling from the client, it’s reinventing the wheel compared to just using an existing websockets library, and it’ll still be inefficient compared to something based around libev or the fiber implementation in the latest release of PHP. But still, interesting problem to solve if you do have nothing but a locked-down legacy server to play with and you’re communicating with it over satellite!
Thanks again for the insights. Yes, short-polling is what I'm also thinking is the path of least resistance here considering that the request and response payload won't be much. It'll return an...
Thanks again for the insights. Yes, short-polling is what I'm also thinking is the path of least resistance here considering that the request and response payload won't be much. It'll return an empty body most of the time.
stream_socket_server is a really great exploration idea, thanks! Pretty sure you can create something like node's event model in PHP with this.
The sleep might not block the CPU, but it will block the PHP process and make it unavailable for reuse by the server, meaning that you need an entire process per user with all the overheads that entails, rather than just a few processes in total that can handle the short polling requests in sequence.
Are you sure about that? What you're saying was perhaps true in the old prefork model of apache but the newer process models like mpm_worker and mpm_event are more intelligent it seems. They most likely won't spawn a new process for a new web request but use existing worker threads for that. This means my original idea of inducing sleeps between the notification fetching loop may still scale as the "slept" threads won't consume as much RAM as entire processes would? Apache also provides configuration variables like ThreadsPerChild and MaxThreads to control the number of these threads which I can theoretically increase to whatever number of requests I want to scale?
For now, both short and long polling are working practically in my testing for my limited number of users, but I'm noting down both these scaling ideas (apache tweaks and PHP stream_socket_server) for future experimentation!
Edit
Here is apparently a very rudimentary implementation of a websockets server in pure PHP.
No worries! It's been interesting to catch up with what's happening in the PHP world nowadays, I haven't had a chance to do so in a little while. I'll never say never, but yeah, I'm reasonably...
No worries! It's been interesting to catch up with what's happening in the PHP world nowadays, I haven't had a chance to do so in a little while.
Are you sure about that?
I'll never say never, but yeah, I'm reasonably certain on this one. It's what I was nodding towards with the two bullet points at the end there, but I can see that they maybe weren't totally clear.
You've got quite a few moving parts, all of which may or may not be independent depending on your config: HTTP connections, Apache processes, Apache threads, PHP processes, and PHP threads.
You can also have multiple HTTP requests over a persistent HTTP connection, but that's not really relevant here since long polling involves keeping a single request open for an extended period of time, so I'll skip it for the sake of clarity.
In the oldest model, prefork and mod_php used a dedicated Apache process for each HTTP connection and a dedicated PHP process for each Apache process. mod_php isn't thread safe, so can only be used with prefork - but this has been considered a legacy configuration for quite some time now, the only reason to use it would be if you're really stuck with mod_php for some reason.
prefork with PHP-FPM will still use an Apache process per connection, but allows the PHP process pool to be scaled independently. You can still have it 1:1 if you want to, but you can also scale up your HTTP concurrency (i.e. more Apache processes) at the expense of latency (i.e. fewer PHP processes, so a given Apache process might have to wait for one to become available). I can't think of a good reason to stick with prefork if you're able to use PHP-FPM.
worker with PHP-FPM will use one or more Apache processes, each of which uses many threads and assigns one thread per connection. This is more efficient than prefork because each thread Apache creates has much lower overhead than an entire new Apache process - some resources can be shared between all threads in a process. PHP, however, still only allows one thread per process* assuming no C extensions are in play, so there will still be a limited pool of PHP processes for those Apache threads to speak to. Sleeping a PHP thread blocks the whole PHP process, effectively removing it from the pool until it completes.
event with PHP-FPM uses one or more Apache process and allows you to efficiently handle more than one HTTP connection per thread, but PHP process pool behaviour remains the same as the previous two configurations. Nginx is similar, at least in the ways that matter for this question.
So you're right that an idle HTTP connection or idle open HTTP request will not necessarily be a problem as long as you're using one of the more modern HTTP server architectures - but with those architectures, PHP capacity and HTTP capacity scale separately. The issue with calling sleep is that it blocks the PHP thread, this blocks a whole PHP process (including memory overhead) because PHP is single threaded, and eventually you run out of space for more PHP processes and everything locks up.
The variables you mentioned are for Apache threads - it's pm.max_children you want to be looking at for the PHP process limit, and that'll be capped by your system RAM even if they're idle.
*Specifically one OS thread, since fibers in PHP 8 can be used to implement green threads in userspace, but that's complicating matters for something that isn't really relevant here.
That's cool! If I'm reading it correctly that's still blocking, so it'll only allow one remote user per PHP process, but there's a non-blocking example in the comments on the stream_socket_server...
Here is apparently a very rudimentary implementation of a websockets server in pure PHP.
That's cool! If I'm reading it correctly that's still blocking, so it'll only allow one remote user per PHP process, but there's a non-blocking example in the comments on the stream_socket_server docs that'll support multiple users on a single process and isn't too much more complicated.
Thanks for the ping! I've done that myself once or twice, it seems fairly easy to mistake the top-level input for a reply box on the bottom comment if you're not thinking about it too much - I...
Thanks for the ping! I've done that myself once or twice, it seems fairly easy to mistake the top-level input for a reply box on the bottom comment if you're not thinking about it too much - I wonder if making the input area or submit button a slightly different colour for top-level comments would be helpful?
You would probably be surprised how often the same happens even on reddit, where the box is at the top. :P But yeah, making it more distinct is probably a good idea anyways. It might reduce the...
You would probably be surprised how often the same happens even on reddit, where the box is at the top. :P
But yeah, making it more distinct is probably a good idea anyways. It might reduce the frequency with which it happens here. So, I added the suggestion to Gitlab: https://gitlab.com/tildes/tildes/-/issues/742
Ah, the joys of UI design! This is exactly why I’ll stick with nice safe backend work wherever I can get away with it… Either way, cheers for adding it to the list.
You would probably be surprised how often the same happens even on reddit, where the box at the top. :P
Ah, the joys of UI design! This is exactly why I’ll stick with nice safe backend work wherever I can get away with it… Either way, cheers for adding it to the list.
I hesitate to be too negative here, but I've got some pretty significant concerns about this one...
Since it depends on a
sleepcall on the server side, I'm not really sure what it's intended to gain over repeated short polling from the client? It'd be the same level of not-quite-realtime, but be radically more scalable since each client would block a PHP process for a few milliseconds every 5 seconds, rather than 300 full seconds at a time. Even assuming an excessively slow DB query and accounting for overhead, repeated polling from the client would allow for more than 50x the throughput on an otherwise identical server configuration.This definitely isn't inevitable. Non-blocking asynchronous IO is pretty standard nowadays - nginx uses it, for example, and then farms those requests out to a fixed pool of PHP workers. In fact, using nginx itself to repeatedly poll those PHP processes while keeping the client connection open would make for an interesting experiment! I don't think it'd be the optimal choice, and I imagine some hacks would be involved to make it happen at all, but it'd be a significant improvement over blocking a PHP process for each user.
Most major languages have at least one mainstream event loop implementation, including PHP itself (Revolt, Amp, or ReactPHP), and while we could probably quibble about minor differences and underlying details that make each one unique, the architecture and usage in all of them tends to be pretty similar to that of V8, which is Node's JS engine. They definitely all make it straightforward to break the thread-per-connection model if you want to.
Any implementation that I'm aware of will use a thread pool and non-blocking IO. I guess if you go right down to the OS level you can argue that something is doing the waiting, but any developer I know would consider that to be misleading at best - you can comfortably support literal millions of concurrent websockets on a single CPU core if they're all just idle.
Thanks for going through the post, Greg, I get the crux of what you're trying to say here.
I'm trying to conserve network bandwidth here. If the polling happened every 5 seconds on client side, there is a concern those concurrent requests might choke/waste the net bandwidth during active use, especially considering all those 30-40 users are in the same LAN/WLAN. The app is still under testing stage and, in fact, it's still being debated whether to host the PHP scripts on the internet cloud or on a local machine in the LAN itself. If that happens, bandwidth concerns may not arise but I think long-polling is still better and more efficient than short?
I don't think the process will be blocked for full 300 seconds. That's why I've included
sleep()in the loop which "idles" the block for 5 seconds at each iteration. Effectively, these sleep chunks should release the CPU and make it idle again for other processes to use. At least in theory, it should scale to a high enough number of requests considering that all processes won't be active at once and most will be in idle state state due to these sleep() chunks?Consider that rotating chair analogy where there are limited number of seats and a slightly higher number of users. But funnily, each user will be pulled up to the roof and stuck there for 5 seconds so that the chair is vacated during that time for other users to use!
An HTTP request like this should only send a few hundred bytes each way, so I really wouldn't worry about it for 40 users. A quick back of the envelope calculation suggests you'd be able to support all of your users on a single 56Kbps dial up connection if you needed to, and if you do use any kind of public internet facing server you'll get more requests than this just from bots sweeping for known vulnerabilities, so I'm pretty confident in saying bandwidth is a non-issue for your use case.
You're right that it is still wasteful in the abstract, and it might eventually become an issue at significant scale, but that's the exact concern that websockets are designed to solve. As it is, if you need a quick and dirty solution without any dependencies, short polling will scale to a few thousand concurrent users on any reasonable modern connection.
In comparison, long polling is limited by your number of concurrent PHP processes. You're right that CPU isn't a major concern with
sleep, but memory absolutely is: it's been years since I last used PHP in production, but from what I'm seeing in docs the memory overhead is still tens of MB. The normal way to handle this is for the web server to keep a limited pool of PHP workers, assign each one to a given incoming request for the minimum time possible, and then pass it along to the next one if there are more requests waiting than available processes. Thesleepmight not block the CPU, but it will block the PHP process and make it unavailable for reuse by the server, meaning that you need an entire process per user with all the overheads that entails, rather than just a few processes in total that can handle the short polling requests in sequence. Even being conservative with the RAM estimations, you're limited to tens or hundreds of users on a server that would otherwise have scaled to thousands.There are two really important things in understanding the tradeoffs here:
So, I’ve kind of nerd sniped myself here and tried to figure out how to solve this efficiently in pure PHP without “cheating” (by an entirely arbitrary definition that I just made up: no libraries, no C extensions, and no fibers).
It looks like the key is
stream_select, which takes a timeout as an upper limit but immediately returns on stream activity if it occurs before that. It gives that lower level non-blocking behaviour that’s otherwise impossible in a single threaded language like PHP. Along with astream_socket_serveryou can basically treat it like a network-interruptiblesleepcall, which in turn allows you to handle multiple connections per process - you can just add them to an array as they come in, even while you’re idling on the loop for an existing connection. Add a bit of calculation to keep track of how long is left on each connection’s 5 second timer each time the loop runs, set your timeout value to the shortest one, and you’ve just built yourself a rudimentary event loop!I wouldn’t call it a good idea: it’s way overcomplicated compared to just repeatedly polling from the client, it’s reinventing the wheel compared to just using an existing websockets library, and it’ll still be inefficient compared to something based around
libevor the fiber implementation in the latest release of PHP. But still, interesting problem to solve if you do have nothing but a locked-down legacy server to play with and you’re communicating with it over satellite!Thanks again for the insights. Yes, short-polling is what I'm also thinking is the path of least resistance here considering that the request and response payload won't be much. It'll return an empty body most of the time.
stream_socket_serveris a really great exploration idea, thanks! Pretty sure you can create something like node's event model in PHP with this.Are you sure about that? What you're saying was perhaps true in the old
preforkmodel of apache but the newer process models like mpm_worker and mpm_event are more intelligent it seems. They most likely won't spawn a new process for a new web request but use existing worker threads for that. This means my original idea of inducing sleeps between the notification fetching loop may still scale as the "slept" threads won't consume as much RAM as entire processes would? Apache also provides configuration variables likeThreadsPerChildandMaxThreadsto control the number of these threads which I can theoretically increase to whatever number of requests I want to scale?For now, both short and long polling are working practically in my testing for my limited number of users, but I'm noting down both these scaling ideas (apache tweaks and PHP stream_socket_server) for future experimentation!
Edit
Here is apparently a very rudimentary implementation of a websockets server in pure PHP.
No worries! It's been interesting to catch up with what's happening in the PHP world nowadays, I haven't had a chance to do so in a little while.
I'll never say never, but yeah, I'm reasonably certain on this one. It's what I was nodding towards with the two bullet points at the end there, but I can see that they maybe weren't totally clear.
You've got quite a few moving parts, all of which may or may not be independent depending on your config: HTTP connections, Apache processes, Apache threads, PHP processes, and PHP threads.
You can also have multiple HTTP requests over a persistent HTTP connection, but that's not really relevant here since long polling involves keeping a single request open for an extended period of time, so I'll skip it for the sake of clarity.
preforkandmod_phpused a dedicated Apache process for each HTTP connection and a dedicated PHP process for each Apache process.mod_phpisn't thread safe, so can only be used withprefork- but this has been considered a legacy configuration for quite some time now, the only reason to use it would be if you're really stuck withmod_phpfor some reason.preforkwithPHP-FPMwill still use an Apache process per connection, but allows the PHP process pool to be scaled independently. You can still have it 1:1 if you want to, but you can also scale up your HTTP concurrency (i.e. more Apache processes) at the expense of latency (i.e. fewer PHP processes, so a given Apache process might have to wait for one to become available). I can't think of a good reason to stick withpreforkif you're able to usePHP-FPM.workerwithPHP-FPMwill use one or more Apache processes, each of which uses many threads and assigns one thread per connection. This is more efficient thanpreforkbecause each thread Apache creates has much lower overhead than an entire new Apache process - some resources can be shared between all threads in a process. PHP, however, still only allows one thread per process* assuming no C extensions are in play, so there will still be a limited pool of PHP processes for those Apache threads to speak to. Sleeping a PHP thread blocks the whole PHP process, effectively removing it from the pool until it completes.eventwithPHP-FPMuses one or more Apache process and allows you to efficiently handle more than one HTTP connection per thread, but PHP process pool behaviour remains the same as the previous two configurations. Nginx is similar, at least in the ways that matter for this question.So you're right that an idle HTTP connection or idle open HTTP request will not necessarily be a problem as long as you're using one of the more modern HTTP server architectures - but with those architectures, PHP capacity and HTTP capacity scale separately. The issue with calling
sleepis that it blocks the PHP thread, this blocks a whole PHP process (including memory overhead) because PHP is single threaded, and eventually you run out of space for more PHP processes and everything locks up.The variables you mentioned are for Apache threads - it's
pm.max_childrenyou want to be looking at for the PHP process limit, and that'll be capped by your system RAM even if they're idle.*Specifically one OS thread, since fibers in PHP 8 can be used to implement green threads in userspace, but that's complicating matters for something that isn't really relevant here.
That's cool! If I'm reading it correctly that's still blocking, so it'll only allow one remote user per PHP process, but there's a non-blocking example in the comments on the
stream_socket_serverdocs that'll support multiple users on a single process and isn't too much more complicated.@Greg. Since you accidentally made a new top-level comment instead of a reply to them, noble_pleb.
Thanks for the ping! I've done that myself once or twice, it seems fairly easy to mistake the top-level input for a reply box on the bottom comment if you're not thinking about it too much - I wonder if making the input area or submit button a slightly different colour for top-level comments would be helpful?
You would probably be surprised how often the same happens even on reddit, where the box is at the top. :P
But yeah, making it more distinct is probably a good idea anyways. It might reduce the frequency with which it happens here. So, I added the suggestion to Gitlab: https://gitlab.com/tildes/tildes/-/issues/742
Ah, the joys of UI design! This is exactly why I’ll stick with nice safe backend work wherever I can get away with it… Either way, cheers for adding it to the list.