Hi, I made a thing! This is by far the most work I've ever sunk into a side project; I've been working on this thing for over two years, and I'm super proud of it, even though there's still a lot...
Hi, I made a thing! This is by far the most work I've ever sunk into a side project; I've been working on this thing for over two years, and I'm super proud of it, even though there's still a lot more to do!
Storyteller is a self-hosted platform for ebooks with synced narration. This is basically self-hosted WhisperSync, for anyone familiar with that Amazon product.
It's currently made up of two self-hostable backend systems and a mobile app for reading and listening to the books it produces. Technically it uses an open spec, EPUB 3's "Media Overlay", for syncing the narration, but very few ebook apps actually support Media Overlays, and even fewer work well and have nice interfaces.
The mobile app is available on the Apple App Store as "Storyteller Reader", and I plan to release it for Android as well early next year.
Anyway, I hope someone finds this interesting or useful!
Very cool! I've always struggled with reading books thanks to aphantasia, and yet struggle with audiobooks because I need to look at the words. So this seems to be exactly what I was looking for!
Very cool! I've always struggled with reading books thanks to aphantasia, and yet struggle with audiobooks because I need to look at the words. So this seems to be exactly what I was looking for!
I hope it can be something useful for you! If you open a book and start playing from the mini-player, it'll highlight each sentence as it's read aloud. If you're not already familiar with self...
I hope it can be something useful for you! If you open a book and start playing from the mini-player, it'll highlight each sentence as it's read aloud. If you're not already familiar with self hosting, the setup might be a little bit overwhelming; please don't hesitate to reach out if you have any questions!
Hi, I worked with someone a year or two ago, perhaps you, on a narration technology like this for our ancient history videos on YouTube. I continue to be excited by the promise of this work,...
Hi, I worked with someone a year or two ago, perhaps you, on a narration technology like this for our ancient history videos on YouTube. I continue to be excited by the promise of this work, especially with dense and challenging texts. I also can see it be very useful with technical and medical narration in particular.
This looks great! I wasn't aware of Media Overlay, but now I'm hoping it gets widely adopted. Were there any parts of the project that were a lot more challenging than you expected?
This looks great! I wasn't aware of Media Overlay, but now I'm hoping it gets widely adopted.
Were there any parts of the project that were a lot more challenging than you expected?
Thank you! You and me both, for sure. Honestly, almost every part of this has been... pretty challenging! The one piece that I was most comfortable with, building the web UI for administrating the...
Thank you! You and me both, for sure.
Honestly, almost every part of this has been... pretty challenging! The one piece that I was most comfortable with, building the web UI for administrating the server, was also by far the smallest and simplest part of the project.
The forced alignment was months and months and months of tweaking and testing and tweaking and testing. I wrote up a description of the algorithm I ended up with here.
Building the reader was also very challenging, though now that it's working it's incredibly rewarding. I had to do a lot of close readings of the EPUB spec and even more truly chaotic micromanagement of mobile WebViews in order to get the reader experience exactly how I wanted it, and there's honestly still more to do! This is also my first mobile app, though using React Native helped a lot, since I'm very comfortable with React.
Wow, yeah, there are a lot of moving parts here! I can see why they would be very rewarding to figure out, and I hope the remaining work is the same! I've been thinking about improving my epub...
Wow, yeah, there are a lot of moving parts here! I can see why they would be very rewarding to figure out, and I hope the remaining work is the same!
I've been thinking about improving my epub flow, so I may try to set this up once I have some time.
I've been using syncabook with a custom script that converts the text into SRT subtitles on a "video" made of only the book's cover and the book's audio. I'll have to check out your solution also,...
I've been using syncabook with a custom script that converts the text into SRT subtitles on a "video" made of only the book's cover and the book's audio. I'll have to check out your solution also, it looks way more... designed for this instead of a hodgepodge of hacks :D Great job.
Great project! Do you have any recommendations on how to run this on a cloud provider, e.g. AWS ECS, for folks who don't maintain a home server? Or is this just the sort of thing that's better...
Great project! Do you have any recommendations on how to run this on a cloud provider, e.g. AWS ECS, for folks who don't maintain a home server? Or is this just the sort of thing that's better hosted privately?
Thank you!! You should be able to run this with a cloud provider for sure, and it should work with AWS ECS (I'm not super familiar with ECS but I've used analogous services in GCP and...
Thank you!!
You should be able to run this with a cloud provider for sure, and it should work with AWS ECS (I'm not super familiar with ECS but I've used analogous services in GCP and DigitalOcean).
You'll need a data volume for to bind to the /data path on the API container. It doesn't need to be massive or anything, but expect 1 to 2 GB per book (which, I guess, depending on the size of your library, could actually end up pretty massive)
Make sure that your environment variables are sensible. This will hopefully get simpler in the next release, but for now:
On the API, you need to set STORYTELLER_ALLOWED_ORIGINS to the url that you will be accessing the web interface from in the browser.
On the web service, you need to set STORYTELLER_API_HOST to a url that both the bockend (i.e. the web container) and the frontend (i.e. your browser) can access. If that's challenging/not an option, you can use the PUBLIC_STORYTELLER_API_HOST variable to specify an alternate url to access the API from the browser
You'll want a somewhat beefy machine for the API; the transcription and syncing processes are very CPU intensive and can take a while to complete
If your plan is to expose this to the public internet (that should be fine!), make sure you're using HTTPS only!
Hi, I made a thing! This is by far the most work I've ever sunk into a side project; I've been working on this thing for over two years, and I'm super proud of it, even though there's still a lot more to do!
Storyteller is a self-hosted platform for ebooks with synced narration. This is basically self-hosted WhisperSync, for anyone familiar with that Amazon product.
It's currently made up of two self-hostable backend systems and a mobile app for reading and listening to the books it produces. Technically it uses an open spec, EPUB 3's "Media Overlay", for syncing the narration, but very few ebook apps actually support Media Overlays, and even fewer work well and have nice interfaces.
The mobile app is available on the Apple App Store as "Storyteller Reader", and I plan to release it for Android as well early next year.
Anyway, I hope someone finds this interesting or useful!
Very cool! I've always struggled with reading books thanks to aphantasia, and yet struggle with audiobooks because I need to look at the words. So this seems to be exactly what I was looking for!
I hope it can be something useful for you! If you open a book and start playing from the mini-player, it'll highlight each sentence as it's read aloud. If you're not already familiar with self hosting, the setup might be a little bit overwhelming; please don't hesitate to reach out if you have any questions!
I appreciate you. I'll have to wait for the Android release, but I'm very excited
All the more reason to get that out the door, then!
Hi, I worked with someone a year or two ago, perhaps you, on a narration technology like this for our ancient history videos on YouTube. I continue to be excited by the promise of this work, especially with dense and challenging texts. I also can see it be very useful with technical and medical narration in particular.
This looks great! I wasn't aware of Media Overlay, but now I'm hoping it gets widely adopted.
Were there any parts of the project that were a lot more challenging than you expected?
Thank you! You and me both, for sure.
Honestly, almost every part of this has been... pretty challenging! The one piece that I was most comfortable with, building the web UI for administrating the server, was also by far the smallest and simplest part of the project.
The forced alignment was months and months and months of tweaking and testing and tweaking and testing. I wrote up a description of the algorithm I ended up with here.
Building the reader was also very challenging, though now that it's working it's incredibly rewarding. I had to do a lot of close readings of the EPUB spec and even more truly chaotic micromanagement of mobile WebViews in order to get the reader experience exactly how I wanted it, and there's honestly still more to do! This is also my first mobile app, though using React Native helped a lot, since I'm very comfortable with React.
Wow, yeah, there are a lot of moving parts here! I can see why they would be very rewarding to figure out, and I hope the remaining work is the same!
I've been thinking about improving my epub flow, so I may try to set this up once I have some time.
Please let me know if you do, and don't hesitate to reach out if you run into any issues!
I've been using syncabook with a custom script that converts the text into SRT subtitles on a "video" made of only the book's cover and the book's audio. I'll have to check out your solution also, it looks way more... designed for this instead of a hodgepodge of hacks :D Great job.
Oh wow! Yeah I'm familiar with syncabook, it's a cool project! I hope Storyteller can make your life a bit simpler, though :)
Great project! Do you have any recommendations on how to run this on a cloud provider, e.g. AWS ECS, for folks who don't maintain a home server? Or is this just the sort of thing that's better hosted privately?
Thank you!!
You should be able to run this with a cloud provider for sure, and it should work with AWS ECS (I'm not super familiar with ECS but I've used analogous services in GCP and DigitalOcean).
/datapath on the API container. It doesn't need to be massive or anything, but expect 1 to 2 GB per book (which, I guess, depending on the size of your library, could actually end up pretty massive)STORYTELLER_ALLOWED_ORIGINSto the url that you will be accessing the web interface from in the browser.STORYTELLER_API_HOSTto a url that both the bockend (i.e. the web container) and the frontend (i.e. your browser) can access. If that's challenging/not an option, you can use thePUBLIC_STORYTELLER_API_HOSTvariable to specify an alternate url to access the API from the browserIf your plan is to expose this to the public internet (that should be fine!), make sure you're using HTTPS only!
Awesome, thanks for the tips!