This is the first release with a new datatype called Redis Streams. Streams are "Kafka done right" and look really, really promising for a lot of use cases where Redis pub/sub is used currently.
This is the first release with a new datatype called Redis Streams. Streams are "Kafka done right" and look really, really promising for a lot of use cases where Redis pub/sub is used currently.
Nice, I've been waiting for streams to make it to the release version. I haven't fully looked into it yet, but I'm hoping that I'll be able to use streams on Tildes to replace rabbitmq. It would...
Nice, I've been waiting for streams to make it to the release version. I haven't fully looked into it yet, but I'm hoping that I'll be able to use streams on Tildes to replace rabbitmq. It would be nice to drop one more system out of the infrastructure, especially since it requires installing some other things like Erlang that I wouldn't otherwise need.
If anyone else is interested in reading more about streams, this was a good post from a while ago: https://brandur.org/redis-streams
IIRC currently there's a bridge that turns Postgres NOTIFYs into RabbitMQ messages...are you thinking it'd be the same thing, but to a Redis stream? pg-amqp-bridge is simpler than I thought it...
IIRC currently there's a bridge that turns Postgres NOTIFYs into RabbitMQ messages...are you thinking it'd be the same thing, but to a Redis stream? pg-amqp-bridge is simpler than I thought it would be, only a couple hundred lines of Rust. I suppose a similar thing for Redis would be fairly straightforward.
Yeah, exactly. I used pg-amqp-bridge because someone had already written it, but like you said, it's really simple and duplicating it (even in Python, if I don't want to bother with introducing a...
Yeah, exactly. I used pg-amqp-bridge because someone had already written it, but like you said, it's really simple and duplicating it (even in Python, if I don't want to bother with introducing a different language) should be straightforward.
It'll probably be more work overall to rewrite the consumers to be able to read from streams, but that shouldn't be too bad either, I don't think.
"Kafka done right" (assuming there is anything wrong with it) already exists in the form of Kinesis (which is quite impressive - my favourite AWS product), but it's good to see a simpler, more...
"Kafka done right" (assuming there is anything wrong with it) already exists in the form of Kinesis (which is quite impressive - my favourite AWS product), but it's good to see a simpler, more minimal and open source alternative.
I used to work for AWS, the Kinesis team was one floor down from me. They're a very smart bunch of engineers, but I wouldn't put it in the same class as Redis. AWS has two types of services -...
I used to work for AWS, the Kinesis team was one floor down from me. They're a very smart bunch of engineers, but I wouldn't put it in the same class as Redis.
AWS has two types of services - hosted generic stuff (EC2, S3, RDS, Route 53) and bespoke custom services (SQS, DynamoDB, Kinesis, etc). The former are things you could run in a private datacenter, or on Google Cloud / Azure / whatever; the latter are AWS-specific and couldn't be used anywhere else.
The general business strategy of AWS is that they want you to use the AWS-specific stuff, so they offer it as a loss leader, with the goal of locking you in to AWS so you'll pay their inflated prices for the more commoditized services.
I didn't mean to, that's why I said "simpler, more minimal" alternative. I meant Redis streams might be a viable alternative to Kinesis or Kafka if all you need is a relatively low traffic,...
I wouldn't put [Kinesis] in the same class as Redis
I didn't mean to, that's why I said "simpler, more minimal" alternative. I meant Redis streams might be a viable alternative to Kinesis or Kafka if all you need is a relatively low traffic, simple, multiple input / multiple output stream with checkpoints (like Kinesis' shard iterators).
The general business strategy of AWS is that they want you to use the AWS-specific stuff, so they offer it as a loss leader, with the goal of locking you in to AWS so you'll pay their inflated prices for the more commoditized services.
That's true and usually it's easy to avoid the lock-in stuff (e.g running MySQL in EC2 isn't too difficult over using Aurora) but sometimes the value add is significant. I think Kinesis is a good example of that. It allows you to get a Kafka-like experience without making you do cluster management.
Looks like it's the same thing as the initial lolwut announcement. Tested with: docker run -d --name redis5 redis:5.0 docker exec -it redis5 redis-cli lolwut Here's the output: http://ix.io/1pzX...
This is the first release with a new datatype called Redis Streams. Streams are "Kafka done right" and look really, really promising for a lot of use cases where Redis pub/sub is used currently.
Nice, I've been waiting for streams to make it to the release version. I haven't fully looked into it yet, but I'm hoping that I'll be able to use streams on Tildes to replace rabbitmq. It would be nice to drop one more system out of the infrastructure, especially since it requires installing some other things like Erlang that I wouldn't otherwise need.
If anyone else is interested in reading more about streams, this was a good post from a while ago: https://brandur.org/redis-streams
IIRC currently there's a bridge that turns Postgres
NOTIFYs into RabbitMQ messages...are you thinking it'd be the same thing, but to a Redis stream? pg-amqp-bridge is simpler than I thought it would be, only a couple hundred lines of Rust. I suppose a similar thing for Redis would be fairly straightforward.Yeah, exactly. I used pg-amqp-bridge because someone had already written it, but like you said, it's really simple and duplicating it (even in Python, if I don't want to bother with introducing a different language) should be straightforward.
It'll probably be more work overall to rewrite the consumers to be able to read from streams, but that shouldn't be too bad either, I don't think.
Any reason you're not using SMS on AWS or PubSub on GCE (clearly I didn't search who you're currently hosted on before asking).
I'm not using cloud hosting.
"Kafka done right" (assuming there is anything wrong with it) already exists in the form of Kinesis (which is quite impressive - my favourite AWS product), but it's good to see a simpler, more minimal and open source alternative.
I used to work for AWS, the Kinesis team was one floor down from me. They're a very smart bunch of engineers, but I wouldn't put it in the same class as Redis.
AWS has two types of services - hosted generic stuff (EC2, S3, RDS, Route 53) and bespoke custom services (SQS, DynamoDB, Kinesis, etc). The former are things you could run in a private datacenter, or on Google Cloud / Azure / whatever; the latter are AWS-specific and couldn't be used anywhere else.
The general business strategy of AWS is that they want you to use the AWS-specific stuff, so they offer it as a loss leader, with the goal of locking you in to AWS so you'll pay their inflated prices for the more commoditized services.
I didn't mean to, that's why I said "simpler, more minimal" alternative. I meant Redis streams might be a viable alternative to Kinesis or Kafka if all you need is a relatively low traffic, simple, multiple input / multiple output stream with checkpoints (like Kinesis' shard iterators).
That's true and usually it's easy to avoid the lock-in stuff (e.g running MySQL in EC2 isn't too difficult over using Aurora) but sometimes the value add is significant. I think Kinesis is a good example of that. It allows you to get a Kafka-like experience without making you do cluster management.
Opensource platform vs proprietary PaaS though. Depending on how you define "done right" Kinesis might not qualify simply because of that.
Thanks for the news! Any word on what the new lolwut easter egg is for this release?
Looks like it's the same thing as the initial lolwut announcement.
Tested with:
Here's the output: http://ix.io/1pzX (looks much better in my console, though)
Ah, thanks for testing. It's a completely silly thing but I'm kinda disappointed that it wasn't changed for the new release, as promised.
The announcement was only a month ago - I thought that one was written specifically for this release?
I was wondering the same thing.