This is one of those things that I think every software dev learns the hard way, and it's nice to see Xe spell it out so cleanly. I think the best summary of "why you shouldn't use time-delays to...
This is one of those things that I think every software dev learns the hard way, and it's nice to see Xe spell it out so cleanly. I think the best summary of "why you shouldn't use time-delays to synchronize things" is the line:
Waiting one second does not make the service ready. The service being ready makes it ready.
I think it's okay in specific cases. An example would be polling - make an API call, wait 5 seconds do it again, etc To be clear I don't think polling is always the right thing to do, but...
I think it's okay in specific cases. An example would be polling - make an API call, wait 5 seconds do it again, etc
To be clear I don't think polling is always the right thing to do, but sometimes it is unavoidable
Polling doesn't make a program concurrent. All the API calls are in a single thread, and the polling happens in a loop: do sleep 5 seconds poll server until server is done
Polling doesn't make a program concurrent. All the API calls are in a single thread, and the polling happens in a loop:
do
sleep 5 seconds
poll server
until server is done
Agreed, but polling can exist in a concurrent program - in the context of your example I could kick off a separate thread to poll for data updates You could also make multiple API calls within the...
Agreed, but polling can exist in a concurrent program - in the context of your example I could kick off a separate thread to poll for data updates
You could also make multiple API calls within the polling loop each on a separate thread as long as you block the polling thread until they all complete.
The tricky bit for testing is that you will always have performance concerns and that's all about how much time passed compared to how much time you expect to pass. This is a fuzzy concern. If it...
The tricky bit for testing is that you will always have performance concerns and that's all about how much time passed compared to how much time you expect to pass. This is a fuzzy concern. If it often takes one second for a service to become ready, you might want to investigate that? This makes setting a low timeout seem like the right thing to do.
But let's zoom out a bit. What are tests for? A test suite is a notification service that alerts you to potential problems. We can judge it by how good the notifications are.
Flaky timeout failures in a test suite result in a terrible notification service that sends you random alerts that something might be wrong, without doing much to help you figure out what it is. Maybe it's a problem in production code, but more likely the server running the tests was busier than usual. Not now, test suite!
Notifications about potential performance slowdowns are useful but the notification service needs to be designed differently.
I think it would be useful to write in a test case that you expect a service to take less than a second to start up. When that regularly doesn't happen, some kind of warning makes sense, but there should probably be some other dashboard for it. This falls under performance monitoring for test suites.
When a test fails entirely, seeing some warnings about how long things took versus how long they're expected to take might be helpful diagnostics. Also, if you're running the test interactively, the developer is waiting on it. If they see some warnings about blown performance expectations then they might decide it's never going to finish.
Sometimes that's not true. If you waited long enough, it would finish. I've sometimes gone to lunch and when I got back, the test actually did finish successfully, far later than expected, and that was an important clue that it's not a correctness problem, it's a performance problem. If you stop the test early then you don't get that clue.
You need some sort of timeout for the overall test, though, to detect the case where things have gotten permanently wedged. A test can't tell the difference between "this is making progress, but...
You need some sort of timeout for the overall test, though, to detect the case where things have gotten permanently wedged. A test can't tell the difference between "this is making progress, but very slowly" and "this isn't making progress at all."
Absolutely. The occasional vague hint you might get by waiting longer has a high cost and there are diminishing returns. This makes setting timeouts tricky.
Absolutely. The occasional vague hint you might get by waiting longer has a high cost and there are diminishing returns.
As I said on the red website: Absolutely true as stated, but there is another sense in which time can be used as a synchronisation primitive. To wit, if a sufficiently high-precision, globally...
As I said on the red website:
Absolutely true as stated, but there is another sense in which time can be used as a synchronisation primitive. To wit, if a sufficiently high-precision, globally synchronised clock is available, then its timestamps can be used to latently order events. In effect, what’s needed is that, for events A and B, happens-before(A, B) implies timestamp(A) < timestamp(B); hence, the precision of the clock has to be greater than the communication delay. (Concurrent events might have the same timestamp, or different timestamps; it doesn’t matter, as semantics are preserved in either event, though you do need some deterministic tiebreaker in the former case.)
Of course, actually realising such a clock is impossible, but that hasn’t stopped anybody. Many intel cpus feature an ‘invariant tsc’—a clock which is guaranteed to be synchronised across all cores in a system (including across sockets, in a multi-socket system), and which runs quite fast (3ghz, on my cpu), so it could in principle be used for this purpose. I believe there is an analogue in the distributed world which actually sees some use, though I forget the name. (Somebody says that google spanner works this way.)
To formalise this a bit more: we can say that sampling the clock and receiving a timestamp t synchronises-with everybody else who sampled the clock and received a timestamp <t. Whereas in the example, the assumption underlying the faulty code was an assumption about the timing of future events, which no memory/execution model I know of accounts for at all (eg c++ only says that threads will make progress ‘eventually’).
As soon as I saw the first Go snipped I thought "oh, this needs a wait group". Then I saw the other snippet where net.Listen was simply lifted outside of the goroutine's scope... I'm not sure I...
As soon as I saw the first Go snipped I thought "oh, this needs a wait group". Then I saw the other snippet where net.Listen was simply lifted outside of the goroutine's scope... I'm not sure I like what this says about me.
At work, our codebase is pretty good about avoiding time synchronization, but there's one situation where it keeps popping up, which is: there's another process running that needs to finish, and will take some unspecified amount of time, that we don't have visibility into. For example:
An integration test sends several messages through a mocked pubsub interface, then checks that they were sent. But pubsub is running in a different process, so we just wait a bit before checking.
We run docker commands to (a) start the local container with postgres, and (b) start another container that runs migrations on the newly created container. Here, the migration runner needs to wait until the postgres container is up and accepting connections, but we don't have an interface to that. So we essentially poll once time.Sleep(1 * time.Second).
I don't have good solutions for these cases, but what they seem to have in common is that they require bidirectional communication. We need the "other" service to tell us when something is ready or complete, so we're basically stuck polling, and polling is time synchronization at its core.
This is one of those things that I think every software dev learns the hard way, and it's nice to see Xe spell it out so cleanly. I think the best summary of "why you shouldn't use time-delays to synchronize things" is the line:
Wait so you mean all those
I have in my code are wrong?!
Calling sleep() is almost always the wrong thing to do in any serious concurrent program.
I think it's okay in specific cases. An example would be polling - make an API call, wait 5 seconds do it again, etc
To be clear I don't think polling is always the right thing to do, but sometimes it is unavoidable
Polling doesn't make a program concurrent. All the API calls are in a single thread, and the polling happens in a loop:
Agreed, but polling can exist in a concurrent program - in the context of your example I could kick off a separate thread to poll for data updates
You could also make multiple API calls within the polling loop each on a separate thread as long as you block the polling thread until they all complete.
The tricky bit for testing is that you will always have performance concerns and that's all about how much time passed compared to how much time you expect to pass. This is a fuzzy concern. If it often takes one second for a service to become ready, you might want to investigate that? This makes setting a low timeout seem like the right thing to do.
But let's zoom out a bit. What are tests for? A test suite is a notification service that alerts you to potential problems. We can judge it by how good the notifications are.
Flaky timeout failures in a test suite result in a terrible notification service that sends you random alerts that something might be wrong, without doing much to help you figure out what it is. Maybe it's a problem in production code, but more likely the server running the tests was busier than usual. Not now, test suite!
Notifications about potential performance slowdowns are useful but the notification service needs to be designed differently.
I think it would be useful to write in a test case that you expect a service to take less than a second to start up. When that regularly doesn't happen, some kind of warning makes sense, but there should probably be some other dashboard for it. This falls under performance monitoring for test suites.
When a test fails entirely, seeing some warnings about how long things took versus how long they're expected to take might be helpful diagnostics. Also, if you're running the test interactively, the developer is waiting on it. If they see some warnings about blown performance expectations then they might decide it's never going to finish.
Sometimes that's not true. If you waited long enough, it would finish. I've sometimes gone to lunch and when I got back, the test actually did finish successfully, far later than expected, and that was an important clue that it's not a correctness problem, it's a performance problem. If you stop the test early then you don't get that clue.
You need some sort of timeout for the overall test, though, to detect the case where things have gotten permanently wedged. A test can't tell the difference between "this is making progress, but very slowly" and "this isn't making progress at all."
Absolutely. The occasional vague hint you might get by waiting longer has a high cost and there are diminishing returns.
This makes setting timeouts tricky.
As I said on the red website:
Absolutely true as stated, but there is another sense in which time can be used as a synchronisation primitive. To wit, if a sufficiently high-precision, globally synchronised clock is available, then its timestamps can be used to latently order events. In effect, what’s needed is that, for events A and B, happens-before(A, B) implies timestamp(A) < timestamp(B); hence, the precision of the clock has to be greater than the communication delay. (Concurrent events might have the same timestamp, or different timestamps; it doesn’t matter, as semantics are preserved in either event, though you do need some deterministic tiebreaker in the former case.)
Of course, actually realising such a clock is impossible, but that hasn’t stopped anybody. Many intel cpus feature an ‘invariant tsc’—a clock which is guaranteed to be synchronised across all cores in a system (including across sockets, in a multi-socket system), and which runs quite fast (3ghz, on my cpu), so it could in principle be used for this purpose. I believe there is an analogue in the distributed world which actually sees some use, though I forget the name. (Somebody says that google spanner works this way.)
To formalise this a bit more: we can say that sampling the clock and receiving a timestamp t synchronises-with everybody else who sampled the clock and received a timestamp <t. Whereas in the example, the assumption underlying the faulty code was an assumption about the timing of future events, which no memory/execution model I know of accounts for at all (eg c++ only says that threads will make progress ‘eventually’).
As soon as I saw the first Go snipped I thought "oh, this needs a wait group". Then I saw the other snippet where
net.Listenwas simply lifted outside of the goroutine's scope... I'm not sure I like what this says about me.At work, our codebase is pretty good about avoiding time synchronization, but there's one situation where it keeps popping up, which is: there's another process running that needs to finish, and will take some unspecified amount of time, that we don't have visibility into. For example:
time.Sleep(1 * time.Second).I don't have good solutions for these cases, but what they seem to have in common is that they require bidirectional communication. We need the "other" service to tell us when something is ready or complete, so we're basically stuck polling, and polling is time synchronization at its core.