It’s a pity Calibre to date refuses to be refactored into a self-hosted service.
The core logic should be portable, with the app just being an interface to it, but no, the entire project is so much spaghetti it would feed the entire boot for over a year… such a shame.
I tried Kavita and immediately recoiled at the fact that basic features like progress sync or metadata matching are behind a paywall - literally features that don’t cost the developers anything, while having open, active bug reports going back a year on these “premium” features.
Progress sync works fine for me in KOReader with OPDS. Progress Sync Scrobble (to third-parties) is the Kavita+ feature.
My understanding was the Kavita+ items are things to do with third-party services and meta data providers that are an API/cost-based service to the dev. That being said I don’t use any of those features.
which is fucking useless for actual progress sync of books because it doesn’t handle concurrency (multiple readers reading the same content, potentially offline), and more importantly, modern ebook formats have no concept of “page” in transit. Oh, you read page 10? Awesome! Now do tell, is it page 10 on a 5" 800x480 eink display with 48px font size and giant margins/lineheights/word paddings, or is it page 10 on a 13" display of 2480x1860 resolution with 11px font size and barely any margins? Since you’ll get wildly different results in both cases, and OPDS doesn’t really allow for adapting this simple integer to a precise position.
No, for that you require a proper locator scheme, something OPDS doesn’t provide and cannot enforce.
Page based progress is fine for fixed format publications - comics, PDF/DOCX files, etc., but that approach breaks irreparably the moment you switch to dynamically formatted content. In case of EPUV/MOBI/the various Kindle formats, you want to determine the reader’s position based on the first and last paragraph/sentence visible on the reader and correlate that to a position within the actual files of the book, which is actually dynamic, as it can be resolved regardless if it’s XML formatted EPUB or if you dumbed the book down to a simple TXT file.
So no, OPDS’s PSE is at best a stopgap solution for syncing progress.
I’ve looked into Kavita before and it looks good, just need to figure out a way I can wirelessly connect to it using KOreader on my Kindle to transfer books and sync reading progress
I see, that’s good to hear, since KOreader has a direct integration with Calibre, when I connect it to my server it shows up as a external device in Calibre and I can select multiple books in Calibre and directly send to the Kindle in one click which I find more convenient than navigating a OPDS catalog from within my slow kindle and downloading books one by one, but maybe in the future when I get a better e-reader I will give Kavita a try.
My workflow is usually to add a book to my Want to Read list in Kavita, then on a reader I can go to that list through OPDS and browse just that list. Makes things much more managable assuming I don’t spam the list.
Agree, though calibre-web exists and runs in a single Docker container. I’ve been using it for a few years, and it’s great.
Sure its a whole Linux server under the hood just to run Calibre and the services required to give it a web interface and API for reading apps - making it way bigger than it needs to be - but it does the job.
A docker container is preferred, but again, CW isn’t Calibre. Same database but completely different management system + also lacking a lot of the sync opportunities.
The issue is that there’s no open protocol for library syncing. It doesn’t exist because all big players (Amazon, Kobo/Rakuten, B&N, etc.) have their own proprietary system, and need no open alternatives.
OPDS is a thing but it’s meant to replicate a physical library (one you can walk into) in behaviour and approach, not a personal library (list all books I have and give me easy access to them). It’s essentially just an RSS-style feed that has no defined structure, thus isn’t software navigable - e.g. there’s no guarantee you can list all book series, or all authors, and most implementations usually give you very roughly defined “recently added”, or “hot now” book lists…
I’ve actually been working on a solution for this, something that provides an almost Kindle library experience (see all your books from a remote server, sync down the remote ebook file, sync back read progress, filter/search based on book properties, etc.), while being flexible enough for non-readers applications as well. But I haven’t even gotten to the point where I can define the API contract properly, let alone the backing database and mapping to Calibre. Honestly at this stage I feel like the best approach is starting from scratch, establishing modern requirements, and going from there.
It’s a pity Calibre to date refuses to be refactored into a self-hosted service.
The core logic should be portable, with the app just being an interface to it, but no, the entire project is so much spaghetti it would feed the entire boot for over a year… such a shame.
I switched. Kavita is the new hotness.
I found it for comics, but realized it handled books as well as Caliber does, in a modern interface with OPDS support.
I tried Kavita and immediately recoiled at the fact that basic features like progress sync or metadata matching are behind a paywall - literally features that don’t cost the developers anything, while having open, active bug reports going back a year on these “premium” features.
All while licensing the code under GPLv3…
Progress sync works fine for me in KOReader with OPDS. Progress Sync Scrobble (to third-parties) is the Kavita+ feature.
My understanding was the Kavita+ items are things to do with third-party services and meta data providers that are an API/cost-based service to the dev. That being said I don’t use any of those features.
OPDS doesn’t do progress sync, at all… you’re running something else there if that works for you.
https://anansi-project.github.io/docs/opds-pse/specs/v1.2
They use the PS extension. I believe Komga and Kavita maintain the spec now. Reader support for Kavita specifically is in the Wiki.
PSE is page streaming, not progress sync…
PSE is a protocol, how information is used on each side of that protocol is at the developers discretion.
pse:lastRead="10" pse:lastReadDate="2010-01-10T10:01:11Z"which is fucking useless for actual progress sync of books because it doesn’t handle concurrency (multiple readers reading the same content, potentially offline), and more importantly, modern ebook formats have no concept of “page” in transit. Oh, you read page 10? Awesome! Now do tell, is it page 10 on a 5" 800x480 eink display with 48px font size and giant margins/lineheights/word paddings, or is it page 10 on a 13" display of 2480x1860 resolution with 11px font size and barely any margins? Since you’ll get wildly different results in both cases, and OPDS doesn’t really allow for adapting this simple integer to a precise position.
No, for that you require a proper locator scheme, something OPDS doesn’t provide and cannot enforce.
Page based progress is fine for fixed format publications - comics, PDF/DOCX files, etc., but that approach breaks irreparably the moment you switch to dynamically formatted content. In case of EPUV/MOBI/the various Kindle formats, you want to determine the reader’s position based on the first and last paragraph/sentence visible on the reader and correlate that to a position within the actual files of the book, which is actually dynamic, as it can be resolved regardless if it’s XML formatted EPUB or if you dumbed the book down to a simple TXT file.
So no, OPDS’s PSE is at best a stopgap solution for syncing progress.
I’ve looked into Kavita before and it looks good, just need to figure out a way I can wirelessly connect to it using KOreader on my Kindle to transfer books and sync reading progress
The OPDS service works for me, just like on Calibre. I can browse my books from within KOReader.
I see, that’s good to hear, since KOreader has a direct integration with Calibre, when I connect it to my server it shows up as a external device in Calibre and I can select multiple books in Calibre and directly send to the Kindle in one click which I find more convenient than navigating a OPDS catalog from within my slow kindle and downloading books one by one, but maybe in the future when I get a better e-reader I will give Kavita a try.
My workflow is usually to add a book to my Want to Read list in Kavita, then on a reader I can go to that list through OPDS and browse just that list. Makes things much more managable assuming I don’t spam the list.
Agree, though calibre-web exists and runs in a single Docker container. I’ve been using it for a few years, and it’s great.
Sure its a whole Linux server under the hood just to run Calibre and the services required to give it a web interface and API for reading apps - making it way bigger than it needs to be - but it does the job.
Calibre-web isn’t Calibre. It uses the same database, but that’s about it, unless you use the optional conversion mod on the linuxserver container.
A docker container is preferred, but again, CW isn’t Calibre. Same database but completely different management system + also lacking a lot of the sync opportunities.
The issue is that there’s no open protocol for library syncing. It doesn’t exist because all big players (Amazon, Kobo/Rakuten, B&N, etc.) have their own proprietary system, and need no open alternatives.
OPDS is a thing but it’s meant to replicate a physical library (one you can walk into) in behaviour and approach, not a personal library (list all books I have and give me easy access to them). It’s essentially just an RSS-style feed that has no defined structure, thus isn’t software navigable - e.g. there’s no guarantee you can list all book series, or all authors, and most implementations usually give you very roughly defined “recently added”, or “hot now” book lists…
I’ve actually been working on a solution for this, something that provides an almost Kindle library experience (see all your books from a remote server, sync down the remote ebook file, sync back read progress, filter/search based on book properties, etc.), while being flexible enough for non-readers applications as well. But I haven’t even gotten to the point where I can define the API contract properly, let alone the backing database and mapping to Calibre. Honestly at this stage I feel like the best approach is starting from scratch, establishing modern requirements, and going from there.
A docker container is not a whole separate Linux server, it uses the kernel running on the host
Have you seen the apt sources list that CW generates on boot? It’s semantics. 😊
With default runtime, very true. There are other runtimes that can be used that provide better isolation like gVisor, kata, firecracker, etc.