[picked the wrong mailing list the first time, sorry for that]
This is amazing! Congrats to the team.
Magnus is raising a good point (thanks!), there is a standard API to integrate catalog into epub reader apps. It's called OPDS [1] and some Wikisource(s) publishes one [2] since a decade. However, it's not very user friendly and we never made outreach to integrate it natively into apps so it's not much used. This OPDS feed is not based on Wikidata but on metadata present inside of Wikisource.
I still think having our own Wikisource branded app is relevant for outreach and visibility, maybe building on OPDS instead of doing something bespoke would allow to reuse even more code?
Congrats again,
Thomas
[1] https://specs.opds.io/ [2] https://en.wikisource.org/wiki/Help:OPDS
Le vendredi 17 janvier 2025 à 15:37, Magnus Manske magnusmanske@googlemail.com a écrit :
This is great!
Silly question: Considering that the reader app will display epub format, is there no existing (open source) epub reader app that could use Wikisource as a, well, source? I would be surprised if there is not a single app where you can set/add an API URL, we would just have to be compatible with that API spec.
This could avoid a whole load of development and technical debt.
On Fri, Jan 17, 2025 at 7:07 AM Bodhisattwa bodhisattwa.rgkmc@gmail.com wrote:
Hello Wikisourcerers,
We are excited to announce that we have been developing a Wikisource reader app for Android mobile users for the last three months and the beta version of the app is almost ready for testing. The development of the mobile app was a much-awaited request from the Wikisource community and when fully released, hopefully, it will help bring more readers to Wikisource platforms.
Necessary links of the reader app
- A documentation page is in draft on meta wiki here - https://meta.wikimedia.org/wiki/Wikisource_reader_app
- The Github repo of the code is here - https://github.com/cis-india/Wikisource-Reader
Wikidata integration
We have been using the FRBR data model used in the Wikidata WikiProject Books to handle the descriptive metadata needed for filtering and categorizing the contents of the app. To be included in the app, an edition should have
- fully proofread or validated
- transcluded into the main namespace or NS:0
- a Wikidata item with descriptive metadata property like title, author etc.
- Wikisource index page url (P1957) property linked with that item
- Wikisource sitelink for the NS:0 transcluded page linked with that item
- a proofread and/or validated badge linked with that sitelink.
Here are a few example of the book editions which are included in the app
- An example of a Bangla book edition is this one
- A list of all French books to be included in the app can be found here - https://w.wiki/CkMP
Pilot languages and API
In the beta version, we have piloted with English, French and Bangla as these three languages are storing their metadata on Wikidata. We will roll out for other languages gradually. An API was built using Django and deployed on Toolforge which serves a catalogue of books following the data model described above. It periodically runs a set of SPARQL queries to retrieve data, processes that data and updates the database.
Features
We have decided to display the books in epub format which will be generated through WS-export tool. Any issues related to epub generation can be reported here.
Right now, the following features are available
- Browse through a list of works which are completely proofread or validated.
- Filter based on the language or literary form of your choice.
- Search for books by title or author.
- Download books to your local device for offline access.
- Share links to the Wikisource page of each book.
- Read, delete, and track completion percentage of books in your local library.
- Change font size, look up words, and access a dictionary.
- Switch between dark and light themes for comfortable reading.
- Use the app in multiple languages.
Currently, we are working on the following areas
- Internal epub reader which can render the styling of the contents as much as possible
- Design of the contents and metadata within the app
- Fixes here and there
Next steps
In coming days, we will
start conversation with the Wikisource communities to make aware about the app, share the workflow needed to include contents there and get feedback for further development. (We had one call with the Marathi Wikisource community already).
Next community conversation hour
Sunday, 26 January 12:00 – 13:00 UTC
Video call link: https://meet.google.com/khd-qvfy-nsr
discuss with all stakeholders regarding necessary steps.
register for Google Play Store and release the stable version of the app there
release the app in other play stores gradually.
We would like to specially thank Sai Phanindra, who has developed the app as an assignment from CIS-A2K.
Regards, Bodhisattwa (Part-time consultant, CIS-A2K) _______________________________________________ Wikisource-l mailing list -- wikisource-l@lists.wikimedia.org To unsubscribe send an email to wikisource-l-leave@lists.wikimedia.org
This app looks great! Very exciting.
On the topic of OPDS: there was some experimenting a few years ago with the OPDS feeds that ws-export currently produces, and the open source FBReader app which supports them: https://fbreader.org/book-sources
The main issues seemed to be around the actual navigation of the catalogue (people generally don't want to browse books alphabetically by title!). And that when OPDS feeds get too big they should ideally be broken up into subsidiary ones (e.g. by topic, for non-fiction, or genre for fiction). But those are not too hard to solve, and it does sound like building them from Wikidata data would be better (although I know there are disagreements about how much cataloguing to have on Wikidata vs Wikisource — that some Wikisources don't want topic, genre etc. to be decided by Wikidata).
There are various issues still with ws-export's stability, and reliability for exporting large or image-heavy books. I'm hoping to work on some of that this month (or rather, upgrade it to Symfony 7 first and then work on that stuff). I wonder if we should be looking again at caching the fully-generated epubs too (although that implies having fewer build-time options maybe).
On 18/1/25 01:22, thomas@pellissier-tanon.fr wrote:
[picked the wrong mailing list the first time, sorry for that]
This is amazing! Congrats to the team.
Magnus is raising a good point (thanks!), there is a standard API to integrate catalog into epub reader apps. It's called OPDS [1] and some Wikisource(s) publishes one [2] since a decade. However, it's not very user friendly and we never made outreach to integrate it natively into apps so it's not much used. This OPDS feed is not based on Wikidata but on metadata present inside of Wikisource.
I still think having our own Wikisource branded app is relevant for outreach and visibility, maybe building on OPDS instead of doing something bespoke would allow to reuse even more code?
Congrats again,
Thomas
[1] https://specs.opds.io/ [2] https://en.wikisource.org/wiki/Help:OPDS
Le vendredi 17 janvier 2025 à 15:37, Magnus Manske magnusmanske@googlemail.com a écrit :
This is great!
Silly question: Considering that the reader app will display epub format, is there no existing (open source) epub reader app that could use Wikisource as a, well, source? I would be surprised if there is not a single app where you can set/add an API URL, we would just have to be compatible with that API spec.
This could avoid a whole load of development and technical debt.
On Fri, Jan 17, 2025 at 7:07 AM Bodhisattwa bodhisattwa.rgkmc@gmail.com wrote:
Hello Wikisourcerers,
We are excited to announce that we have been developing a Wikisource reader app for Android mobile users for the last three months and the beta version of the app is almost ready for testing. The development of the mobile app was a much-awaited request from the Wikisource community and when fully released, hopefully, it will help bring more readers to Wikisource platforms.
Necessary links of the reader app
- A documentation page is in draft on meta wiki here - https://meta.wikimedia.org/wiki/Wikisource_reader_app
- The Github repo of the code is here - https://github.com/cis-india/Wikisource-Reader
Wikidata integration
We have been using the FRBR data model used in the Wikidata WikiProject Books to handle the descriptive metadata needed for filtering and categorizing the contents of the app. To be included in the app, an edition should have
- fully proofread or validated
- transcluded into the main namespace or NS:0
- a Wikidata item with descriptive metadata property like title, author etc.
- Wikisource index page url (P1957) property linked with that item
- Wikisource sitelink for the NS:0 transcluded page linked with that item
- a proofread and/or validated badge linked with that sitelink.
Here are a few example of the book editions which are included in the app
- An example of a Bangla book edition is this one
- A list of all French books to be included in the app can be found here - https://w.wiki/CkMP
Pilot languages and API
In the beta version, we have piloted with English, French and Bangla as these three languages are storing their metadata on Wikidata. We will roll out for other languages gradually. An API was built using Django and deployed on Toolforge which serves a catalogue of books following the data model described above. It periodically runs a set of SPARQL queries to retrieve data, processes that data and updates the database.
Features
We have decided to display the books in epub format which will be generated through WS-export tool. Any issues related to epub generation can be reported here.
Right now, the following features are available
- Browse through a list of works which are completely proofread or validated.
- Filter based on the language or literary form of your choice.
- Search for books by title or author.
- Download books to your local device for offline access.
- Share links to the Wikisource page of each book.
- Read, delete, and track completion percentage of books in your local library.
- Change font size, look up words, and access a dictionary.
- Switch between dark and light themes for comfortable reading.
- Use the app in multiple languages.
Currently, we are working on the following areas
- Internal epub reader which can render the styling of the contents as much as possible
- Design of the contents and metadata within the app
- Fixes here and there
Next steps
In coming days, we will
start conversation with the Wikisource communities to make aware about the app, share the workflow needed to include contents there and get feedback for further development. (We had one call with the Marathi Wikisource community already).
Next community conversation hour
Sunday, 26 January 12:00 – 13:00 UTC
Video call link: https://meet.google.com/khd-qvfy-nsr
discuss with all stakeholders regarding necessary steps.
register for Google Play Store and release the stable version of the app there
release the app in other play stores gradually.
We would like to specially thank Sai Phanindra, who has developed the app as an assignment from CIS-A2K.
Regards, Bodhisattwa (Part-time consultant, CIS-A2K) _______________________________________________ Wikisource-l mailing list -- wikisource-l@lists.wikimedia.org To unsubscribe send an email to wikisource-l-leave@lists.wikimedia.org
Wikisource-l mailing list -- wikisource-l@lists.wikimedia.org To unsubscribe send an email to wikisource-l-leave@lists.wikimedia.org
Hello Wikisourcerers,
Thank you for all your feedback here, on meta-wiki and on telegram channels. We are keeping note of all of them and having discussions on them internally.
Our current e-book reader, which we are using in the app, has many limitations and does not render the stylings used in Wikisource properly. So, we are replacing it with another open-source ebook reader which has a variety of features and can overcome those limitations. It will take us a few more days to update the new code on Github. For this reason, we are postponing the conversation hour to 2nd February, same time.
- *Next community conversation hour* - Sunday, 2 February 12:00 – 13:00 UTC - Video call link: https://meet.google.com/khd-qvfy-nsr
Andy, we intend to give a demonstration of the app at the upcoming Wikisource conference and we have already requested for a small slot in the session to do that, but it is up to the conference program team to decide as the program submission deadline is already closed. Anyway, if it is not possible through official sessions, we can always demo at the meetups or outside official conference sessions.
Regards, Bodhisattwa
On Sat, 18 Jan 2025 at 07:32, Sam Wilson sam@samwilson.id.au wrote:
This app looks great! Very exciting.
On the topic of OPDS: there was some experimenting a few years ago with the OPDS feeds that ws-export currently produces, and the open source FBReader app which supports them: https://fbreader.org/book-sources
The main issues seemed to be around the actual navigation of the catalogue (people generally don't want to browse books alphabetically by title!). And that when OPDS feeds get too big they should ideally be broken up into subsidiary ones (e.g. by topic, for non-fiction, or genre for fiction). But those are not too hard to solve, and it does sound like building them from Wikidata data would be better (although I know there are disagreements about how much cataloguing to have on Wikidata vs Wikisource — that some Wikisources don't want topic, genre etc. to be decided by Wikidata).
There are various issues still with ws-export's stability, and reliability for exporting large or image-heavy books. I'm hoping to work on some of that this month (or rather, upgrade it to Symfony 7 first and then work on that stuff). I wonder if we should be looking again at caching the fully-generated epubs too (although that implies having fewer build-time options maybe).
On 18/1/25 01:22, thomas@pellissier-tanon.fr wrote:
[picked the wrong mailing list the first time, sorry for that]
This is amazing! Congrats to the team.
Magnus is raising a good point (thanks!), there is a standard API to
integrate catalog into epub reader apps. It's called OPDS [1] and some Wikisource(s) publishes one [2] since a decade. However, it's not very user friendly and we never made outreach to integrate it natively into apps so it's not much used. This OPDS feed is not based on Wikidata but on metadata present inside of Wikisource.
I still think having our own Wikisource branded app is relevant for
outreach and visibility, maybe building on OPDS instead of doing something bespoke would allow to reuse even more code?
Congrats again,
Thomas
[1] https://specs.opds.io/ [2] https://en.wikisource.org/wiki/Help:OPDS
Le vendredi 17 janvier 2025 à 15:37, Magnus Manske <
magnusmanske@googlemail.com> a écrit :
This is great!
Silly question: Considering that the reader app will display epub
format, is there no existing (open source) epub reader app that could use Wikisource as a, well, source? I would be surprised if there is not a single app where you can set/add an API URL, we would just have to be compatible with that API spec.
This could avoid a whole load of development and technical debt.
On Fri, Jan 17, 2025 at 7:07 AM Bodhisattwa <
bodhisattwa.rgkmc@gmail.com> wrote:
Hello Wikisourcerers,
We are excited to announce that we have been developing a Wikisource
reader app for Android mobile users for the last three months and the beta version of the app is almost ready for testing. The development of the mobile app was a much-awaited request from the Wikisource community and when fully released, hopefully, it will help bring more readers to Wikisource platforms.
Necessary links of the reader app
- A documentation page is in draft on meta wiki here -
https://meta.wikimedia.org/wiki/Wikisource_reader_app
- The Github repo of the code is here -
https://github.com/cis-india/Wikisource-Reader
Wikidata integration
We have been using the FRBR data model used in the Wikidata
WikiProject Books to handle the descriptive metadata needed for filtering and categorizing the contents of the app. To be included in the app, an edition should have
- fully proofread or validated
- transcluded into the main namespace or NS:0
- a Wikidata item with descriptive metadata property like title,
author etc.
- Wikisource index page url (P1957) property linked with that item
- Wikisource sitelink for the NS:0 transcluded page linked with
that item
- a proofread and/or validated badge linked with that sitelink.
Here are a few example of the book editions which are included in the
app
- An example of a Bangla book edition is this one
- A list of all French books to be included in the app can be found
here - https://w.wiki/CkMP
Pilot languages and API
In the beta version, we have piloted with English, French and Bangla
as these three languages are storing their metadata on Wikidata. We will roll out for other languages gradually. An API was built using Django and deployed on Toolforge which serves a catalogue of books following the data model described above. It periodically runs a set of SPARQL queries to retrieve data, processes that data and updates the database.
Features
We have decided to display the books in epub format which will be
generated through WS-export tool. Any issues related to epub generation can be reported here.
Right now, the following features are available
- Browse through a list of works which are completely proofread or
validated.
- Filter based on the language or literary form of your choice.
- Search for books by title or author.
- Download books to your local device for offline access.
- Share links to the Wikisource page of each book.
- Read, delete, and track completion percentage of books in your
local library.
- Change font size, look up words, and access a dictionary.
- Switch between dark and light themes for comfortable reading.
- Use the app in multiple languages.
Currently, we are working on the following areas
- Internal epub reader which can render the styling of the contents
as much as possible
- Design of the contents and metadata within the app
- Fixes here and there
Next steps
In coming days, we will
- start conversation with the Wikisource communities to make aware
about the app, share the workflow needed to include contents there and get feedback for further development. (We had one call with the Marathi Wikisource community already).
Next community conversation hour
Sunday, 26 January 12:00 – 13:00 UTC
Video call link: https://meet.google.com/khd-qvfy-nsr
discuss with all stakeholders regarding necessary steps.
register for Google Play Store and release the stable version of
the app there
- release the app in other play stores gradually.
We would like to specially thank Sai Phanindra, who has developed the
app as an assignment from CIS-A2K.
Regards, Bodhisattwa (Part-time consultant, CIS-A2K) _______________________________________________ Wikisource-l mailing list -- wikisource-l@lists.wikimedia.org To unsubscribe send an email to
wikisource-l-leave@lists.wikimedia.org
Wikisource-l mailing list -- wikisource-l@lists.wikimedia.org To unsubscribe send an email to wikisource-l-leave@lists.wikimedia.org
Wikisource-l mailing list -- wikisource-l@lists.wikimedia.org To unsubscribe send an email to wikisource-l-leave@lists.wikimedia.org
Hello Wikisourcerers,
Thanks to Sai's commitment, we have successfully switched to a new reader this week and have resolved most of the rendering issues we were having with the last reader. The new reader is rendering the Wikisource stylings as they are. This new reader also comes with a wide range of other features to improve reader experience like search, navigation, highlight, bookmark, underline, annotation, font color options, font size options, font weight options, text normalisation, theme color options, image color inversion, page margin options, different font options, RTL support etc.
We have also included two more languages into the app as pilot i.e. Spanish and Marathi based on their recent contributions to the Wikidata items for book editions following the FRBR book model used in Wikidata:Books and app workflow. Special thanks to Ignacio, Subodh, Ashwini for the valuable contributions on Wikidata.
This is also a gentle reminder that the community conversation hour will happen today at 12:00 noon UTC as mentioned in the previous email.
See you all there,
Regards, Bodhisattwa
On Thu, 23 Jan 2025 at 12:19, Bodhisattwa bodhisattwa.rgkmc@gmail.com wrote:
Hello Wikisourcerers,
Thank you for all your feedback here, on meta-wiki and on telegram channels. We are keeping note of all of them and having discussions on them internally.
Our current e-book reader, which we are using in the app, has many limitations and does not render the stylings used in Wikisource properly. So, we are replacing it with another open-source ebook reader which has a variety of features and can overcome those limitations. It will take us a few more days to update the new code on Github. For this reason, we are postponing the conversation hour to 2nd February, same time.
- *Next community conversation hour*
- Sunday, 2 February 12:00 – 13:00 UTC
- Video call link: https://meet.google.com/khd-qvfy-nsr
Andy, we intend to give a demonstration of the app at the upcoming Wikisource conference and we have already requested for a small slot in the session to do that, but it is up to the conference program team to decide as the program submission deadline is already closed. Anyway, if it is not possible through official sessions, we can always demo at the meetups or outside official conference sessions.
Regards, Bodhisattwa
On Sat, 18 Jan 2025 at 07:32, Sam Wilson sam@samwilson.id.au wrote:
This app looks great! Very exciting.
On the topic of OPDS: there was some experimenting a few years ago with the OPDS feeds that ws-export currently produces, and the open source FBReader app which supports them: https://fbreader.org/book-sources
The main issues seemed to be around the actual navigation of the catalogue (people generally don't want to browse books alphabetically by title!). And that when OPDS feeds get too big they should ideally be broken up into subsidiary ones (e.g. by topic, for non-fiction, or genre for fiction). But those are not too hard to solve, and it does sound like building them from Wikidata data would be better (although I know there are disagreements about how much cataloguing to have on Wikidata vs Wikisource — that some Wikisources don't want topic, genre etc. to be decided by Wikidata).
There are various issues still with ws-export's stability, and reliability for exporting large or image-heavy books. I'm hoping to work on some of that this month (or rather, upgrade it to Symfony 7 first and then work on that stuff). I wonder if we should be looking again at caching the fully-generated epubs too (although that implies having fewer build-time options maybe).
On 18/1/25 01:22, thomas@pellissier-tanon.fr wrote:
[picked the wrong mailing list the first time, sorry for that]
This is amazing! Congrats to the team.
Magnus is raising a good point (thanks!), there is a standard API to
integrate catalog into epub reader apps. It's called OPDS [1] and some Wikisource(s) publishes one [2] since a decade. However, it's not very user friendly and we never made outreach to integrate it natively into apps so it's not much used. This OPDS feed is not based on Wikidata but on metadata present inside of Wikisource.
I still think having our own Wikisource branded app is relevant for
outreach and visibility, maybe building on OPDS instead of doing something bespoke would allow to reuse even more code?
Congrats again,
Thomas
[1] https://specs.opds.io/ [2] https://en.wikisource.org/wiki/Help:OPDS
Le vendredi 17 janvier 2025 à 15:37, Magnus Manske <
magnusmanske@googlemail.com> a écrit :
This is great!
Silly question: Considering that the reader app will display epub
format, is there no existing (open source) epub reader app that could use Wikisource as a, well, source? I would be surprised if there is not a single app where you can set/add an API URL, we would just have to be compatible with that API spec.
This could avoid a whole load of development and technical debt.
On Fri, Jan 17, 2025 at 7:07 AM Bodhisattwa <
bodhisattwa.rgkmc@gmail.com> wrote:
Hello Wikisourcerers,
We are excited to announce that we have been developing a Wikisource
reader app for Android mobile users for the last three months and the beta version of the app is almost ready for testing. The development of the mobile app was a much-awaited request from the Wikisource community and when fully released, hopefully, it will help bring more readers to Wikisource platforms.
Necessary links of the reader app
- A documentation page is in draft on meta wiki here -
https://meta.wikimedia.org/wiki/Wikisource_reader_app
- The Github repo of the code is here -
https://github.com/cis-india/Wikisource-Reader
Wikidata integration
We have been using the FRBR data model used in the Wikidata
WikiProject Books to handle the descriptive metadata needed for filtering and categorizing the contents of the app. To be included in the app, an edition should have
- fully proofread or validated
- transcluded into the main namespace or NS:0
- a Wikidata item with descriptive metadata property like title,
author etc.
- Wikisource index page url (P1957) property linked with that item
- Wikisource sitelink for the NS:0 transcluded page linked with
that item
- a proofread and/or validated badge linked with that sitelink.
Here are a few example of the book editions which are included in
the app
- An example of a Bangla book edition is this one
- A list of all French books to be included in the app can be
found here - https://w.wiki/CkMP
Pilot languages and API
In the beta version, we have piloted with English, French and Bangla
as these three languages are storing their metadata on Wikidata. We will roll out for other languages gradually. An API was built using Django and deployed on Toolforge which serves a catalogue of books following the data model described above. It periodically runs a set of SPARQL queries to retrieve data, processes that data and updates the database.
Features
We have decided to display the books in epub format which will be
generated through WS-export tool. Any issues related to epub generation can be reported here.
Right now, the following features are available
- Browse through a list of works which are completely proofread or
validated.
- Filter based on the language or literary form of your choice.
- Search for books by title or author.
- Download books to your local device for offline access.
- Share links to the Wikisource page of each book.
- Read, delete, and track completion percentage of books in your
local library.
- Change font size, look up words, and access a dictionary.
- Switch between dark and light themes for comfortable reading.
- Use the app in multiple languages.
Currently, we are working on the following areas
- Internal epub reader which can render the styling of the
contents as much as possible
- Design of the contents and metadata within the app
- Fixes here and there
Next steps
In coming days, we will
- start conversation with the Wikisource communities to make aware
about the app, share the workflow needed to include contents there and get feedback for further development. (We had one call with the Marathi Wikisource community already).
Next community conversation hour
Sunday, 26 January 12:00 – 13:00 UTC
Video call link: https://meet.google.com/khd-qvfy-nsr
discuss with all stakeholders regarding necessary steps.
register for Google Play Store and release the stable version of
the app there
- release the app in other play stores gradually.
We would like to specially thank Sai Phanindra, who has developed
the app as an assignment from CIS-A2K.
Regards, Bodhisattwa (Part-time consultant, CIS-A2K) _______________________________________________ Wikisource-l mailing list -- wikisource-l@lists.wikimedia.org To unsubscribe send an email to
wikisource-l-leave@lists.wikimedia.org
Wikisource-l mailing list -- wikisource-l@lists.wikimedia.org To unsubscribe send an email to wikisource-l-leave@lists.wikimedia.org
Wikisource-l mailing list -- wikisource-l@lists.wikimedia.org To unsubscribe send an email to wikisource-l-leave@lists.wikimedia.org
Hello Wikisourcerers,
We have some announcements to make from our end related to the Wikisource reader app.
-
The Wikisource reader app entirely depends on the WS-export https://ws-export.wmcloud.org/ tool while downloading the books. As the maintainers of the WS-export tool are currently working to make it more robust and dependable than before, we have decided to wait for them to finish their work and slightly postpone the launch of the app on play stores for everyone to use. -
CIS-A2K has sponsored the development of the app. As some of you might know from recent official communications coming from them that they are recently in a critical situation. That jeopardises the future maintenance of the app for the next few years. We have explored a few stable Wikimedia affiliates from different parts of the world who can host and help maintain the app in upcoming years, but as we have received repeated assurances from the current leadership of CIS-A2K team, the app will be hosted by them under their play store account for now, until it cannot be sustained there. The app will also be available on F-Droid. -
We have worked on a few improvements including fetching titles and thumbnails, localising language names, improving text selection options, etc. -
For titles, as discussed on the Meta-Wiki talk page, the app will query in the following manner, -
Title (P1476 https://www.wikidata.org/wiki/Property:P1476) in native language -
label in native language, if the above is not present -
label in multiple language (mul), if the above is not present -
For thumbnails, the app will query in the following manner -
Image (P18 https://www.wikidata.org/wiki/Property:P18) -
title page number (P4714 https://www.wikidata.org/wiki/Property:P4714) qualifier used in P996 https://www.wikidata.org/wiki/Property:P996, if the above is not present -
Apart from the existing language support for Bangla, English, French, Hindi, Marathi, Punjabi, Spanish, Tamil, Telugu, we have added Assamese, Indonesian and Malay after getting requests from some of the participants of the Wikisource conference speaking these languages. The app interface is also available in all these languages to some extent. -
We are also working towards localisation of the app, however it will require some time from our end to host the strings on TranslateWiki. Meanwhile, you can reach out to us if you are interested in assisting us with localisation. -
Now that the core development is done, future work will be done voluntarily. We are more than happy to invite the Wikimedia developer community to send bug reports https://github.com/cis-india/Wikisource-Reader/issues, fixes, and features. We are also looking forward to including app related development in upcoming hackathons and internships. -
We will also be publishing some tutorials and guides to easily structure wikidata items related to works on Wikisource. We hope the community takes interest in improving Wikidata items so that we can bring more works into the app and also into broader access. -
The app now lists 6,577 completed works in 12 languages 🎉 -
Assamese - 23 -
Bangla - 457 -
English - 1785 -
French - 3690 -
Hindi - 1 -
Indonesian - 21 -
Malay - 9 -
Marathi - 65 -
Punjabi - 23 -
Spanish - 345 -
Telugu - 77 -
Tamil - 81
Regards,
Bodhisattwa and Sai Phanindra
On Sun, 2 Feb 2025 at 09:49, Bodhisattwa bodhisattwa.rgkmc@gmail.com wrote:
Hello Wikisourcerers,
Thanks to Sai's commitment, we have successfully switched to a new reader this week and have resolved most of the rendering issues we were having with the last reader. The new reader is rendering the Wikisource stylings as they are. This new reader also comes with a wide range of other features to improve reader experience like search, navigation, highlight, bookmark, underline, annotation, font color options, font size options, font weight options, text normalisation, theme color options, image color inversion, page margin options, different font options, RTL support etc.
We have also included two more languages into the app as pilot i.e. Spanish and Marathi based on their recent contributions to the Wikidata items for book editions following the FRBR book model used in Wikidata:Books and app workflow. Special thanks to Ignacio, Subodh, Ashwini for the valuable contributions on Wikidata.
This is also a gentle reminder that the community conversation hour will happen today at 12:00 noon UTC as mentioned in the previous email.
See you all there,
Regards, Bodhisattwa
On Thu, 23 Jan 2025 at 12:19, Bodhisattwa bodhisattwa.rgkmc@gmail.com wrote:
Hello Wikisourcerers,
Thank you for all your feedback here, on meta-wiki and on telegram channels. We are keeping note of all of them and having discussions on them internally.
Our current e-book reader, which we are using in the app, has many limitations and does not render the stylings used in Wikisource properly. So, we are replacing it with another open-source ebook reader which has a variety of features and can overcome those limitations. It will take us a few more days to update the new code on Github. For this reason, we are postponing the conversation hour to 2nd February, same time.
- *Next community conversation hour*
- Sunday, 2 February 12:00 – 13:00 UTC
- Video call link: https://meet.google.com/khd-qvfy-nsr
Andy, we intend to give a demonstration of the app at the upcoming Wikisource conference and we have already requested for a small slot in the session to do that, but it is up to the conference program team to decide as the program submission deadline is already closed. Anyway, if it is not possible through official sessions, we can always demo at the meetups or outside official conference sessions.
Regards, Bodhisattwa
On Sat, 18 Jan 2025 at 07:32, Sam Wilson sam@samwilson.id.au wrote:
This app looks great! Very exciting.
On the topic of OPDS: there was some experimenting a few years ago with the OPDS feeds that ws-export currently produces, and the open source FBReader app which supports them: https://fbreader.org/book-sources
The main issues seemed to be around the actual navigation of the catalogue (people generally don't want to browse books alphabetically by title!). And that when OPDS feeds get too big they should ideally be broken up into subsidiary ones (e.g. by topic, for non-fiction, or genre for fiction). But those are not too hard to solve, and it does sound like building them from Wikidata data would be better (although I know there are disagreements about how much cataloguing to have on Wikidata vs Wikisource — that some Wikisources don't want topic, genre etc. to be decided by Wikidata).
There are various issues still with ws-export's stability, and reliability for exporting large or image-heavy books. I'm hoping to work on some of that this month (or rather, upgrade it to Symfony 7 first and then work on that stuff). I wonder if we should be looking again at caching the fully-generated epubs too (although that implies having fewer build-time options maybe).
On 18/1/25 01:22, thomas@pellissier-tanon.fr wrote:
[picked the wrong mailing list the first time, sorry for that]
This is amazing! Congrats to the team.
Magnus is raising a good point (thanks!), there is a standard API to
integrate catalog into epub reader apps. It's called OPDS [1] and some Wikisource(s) publishes one [2] since a decade. However, it's not very user friendly and we never made outreach to integrate it natively into apps so it's not much used. This OPDS feed is not based on Wikidata but on metadata present inside of Wikisource.
I still think having our own Wikisource branded app is relevant for
outreach and visibility, maybe building on OPDS instead of doing something bespoke would allow to reuse even more code?
Congrats again,
Thomas
[1] https://specs.opds.io/ [2] https://en.wikisource.org/wiki/Help:OPDS
Le vendredi 17 janvier 2025 à 15:37, Magnus Manske <
magnusmanske@googlemail.com> a écrit :
This is great!
Silly question: Considering that the reader app will display epub
format, is there no existing (open source) epub reader app that could use Wikisource as a, well, source? I would be surprised if there is not a single app where you can set/add an API URL, we would just have to be compatible with that API spec.
This could avoid a whole load of development and technical debt.
On Fri, Jan 17, 2025 at 7:07 AM Bodhisattwa <
bodhisattwa.rgkmc@gmail.com> wrote:
> Hello Wikisourcerers, > > We are excited to announce that we have been developing a
Wikisource reader app for Android mobile users for the last three months and the beta version of the app is almost ready for testing. The development of the mobile app was a much-awaited request from the Wikisource community and when fully released, hopefully, it will help bring more readers to Wikisource platforms.
> > Necessary links of the reader app > > - A documentation page is in draft on meta wiki here -
https://meta.wikimedia.org/wiki/Wikisource_reader_app
> - The Github repo of the code is here -
https://github.com/cis-india/Wikisource-Reader
> > Wikidata integration > > We have been using the FRBR data model used in the Wikidata
WikiProject Books to handle the descriptive metadata needed for filtering and categorizing the contents of the app. To be included in the app, an edition should have
> > - fully proofread or validated > - transcluded into the main namespace or NS:0 > - a Wikidata item with descriptive metadata property like title,
author etc.
> - Wikisource index page url (P1957) property linked with that item > - Wikisource sitelink for the NS:0 transcluded page linked with
that item
> - a proofread and/or validated badge linked with that sitelink. > > Here are a few example of the book editions which are included in
the app
> > - An example of a Bangla book edition is this one > - A list of all French books to be included in the app can be
found here - https://w.wiki/CkMP
> > Pilot languages and API > > In the beta version, we have piloted with English, French and
Bangla as these three languages are storing their metadata on Wikidata. We will roll out for other languages gradually. An API was built using Django and deployed on Toolforge which serves a catalogue of books following the data model described above. It periodically runs a set of SPARQL queries to retrieve data, processes that data and updates the database.
> > - Link - https://wsindex.toolforge.org/books/ > - Repo - https://codeberg.org/ph4ni/wsindex > > Features > > We have decided to display the books in epub format which will be
generated through WS-export tool. Any issues related to epub generation can be reported here.
> > Right now, the following features are available > > 1. Browse through a list of works which are completely proofread
or validated.
> 2. Filter based on the language or literary form of your choice. > 3. Search for books by title or author. > 4. Download books to your local device for offline access. > 5. Share links to the Wikisource page of each book. > 6. Read, delete, and track completion percentage of books in your
local library.
> 7. Change font size, look up words, and access a dictionary. > 8. Switch between dark and light themes for comfortable reading. > 9. Use the app in multiple languages. > > Currently, we are working on the following areas > > - Internal epub reader which can render the styling of the
contents as much as possible
> - Design of the contents and metadata within the app > - Fixes here and there > > Next steps > > > In coming days, we will > > - start conversation with the Wikisource communities to make
aware about the app, share the workflow needed to include contents there and get feedback for further development. (We had one call with the Marathi Wikisource community already).
> > - Next community conversation hour > > - Sunday, 26 January 12:00 – 13:00 UTC > - Video call link: https://meet.google.com/khd-qvfy-nsr > > - discuss with all stakeholders regarding necessary steps. > - register for Google Play Store and release the stable version
of the app there
> - release the app in other play stores gradually. > > We would like to specially thank Sai Phanindra, who has developed
the app as an assignment from CIS-A2K.
> > Regards, > Bodhisattwa > (Part-time consultant, CIS-A2K) > _______________________________________________ > Wikisource-l mailing list -- wikisource-l@lists.wikimedia.org > To unsubscribe send an email to
wikisource-l-leave@lists.wikimedia.org
Wikisource-l mailing list -- wikisource-l@lists.wikimedia.org To unsubscribe send an email to wikisource-l-leave@lists.wikimedia.org
Wikisource-l mailing list -- wikisource-l@lists.wikimedia.org To unsubscribe send an email to wikisource-l-leave@lists.wikimedia.org
Thanks indeed for the detailed infos. //The app will also be available on F-Droid.// Waiting ... As an Indian, I have to say "One more hurdle for Learning ...". I hope that we will get International support.
On Mon, 31 Mar 2025 at 17:46, Bodhisattwa bodhisattwa.rgkmc@gmail.com wrote:
Hello Wikisourcerers,
We have some announcements to make from our end related to the Wikisource reader app.
The Wikisource reader app entirely depends on the WS-export https://ws-export.wmcloud.org/ tool while downloading the books. As the maintainers of the WS-export tool are currently working to make it more robust and dependable than before, we have decided to wait for them to finish their work and slightly postpone the launch of the app on play stores for everyone to use.
CIS-A2K has sponsored the development of the app. As some of you might know from recent official communications coming from them that they are recently in a critical situation. That jeopardises the future maintenance of the app for the next few years. We have explored a few stable Wikimedia affiliates from different parts of the world who can host and help maintain the app in upcoming years, but as we have received repeated assurances from the current leadership of CIS-A2K team, the app will be hosted by them under their play store account for now, until it cannot be sustained there. The app will also be available on F-Droid.
We have worked on a few improvements including fetching titles and thumbnails, localising language names, improving text selection options, etc.
For titles, as discussed on the Meta-Wiki talk page, the app will query in the following manner,
Title (P1476 https://www.wikidata.org/wiki/Property:P1476) in native language
label in native language, if the above is not present
label in multiple language (mul), if the above is not present
For thumbnails, the app will query in the following manner
Image (P18 https://www.wikidata.org/wiki/Property:P18)
title page number (P4714 https://www.wikidata.org/wiki/Property:P4714) qualifier used in P996 https://www.wikidata.org/wiki/Property:P996, if the above is not present
Apart from the existing language support for Bangla, English, French, Hindi, Marathi, Punjabi, Spanish, Tamil, Telugu, we have added Assamese, Indonesian and Malay after getting requests from some of the participants of the Wikisource conference speaking these languages. The app interface is also available in all these languages to some extent.
We are also working towards localisation of the app, however it will require some time from our end to host the strings on TranslateWiki. Meanwhile, you can reach out to us if you are interested in assisting us with localisation.
Now that the core development is done, future work will be done voluntarily. We are more than happy to invite the Wikimedia developer community to send bug reports https://github.com/cis-india/Wikisource-Reader/issues, fixes, and features. We are also looking forward to including app related development in upcoming hackathons and internships.
We will also be publishing some tutorials and guides to easily structure wikidata items related to works on Wikisource. We hope the community takes interest in improving Wikidata items so that we can bring more works into the app and also into broader access.
The app now lists 6,577 completed works in 12 languages 🎉
Assamese - 23
Bangla - 457
English - 1785
French - 3690
Hindi - 1
Indonesian - 21
Malay - 9
Marathi - 65
Punjabi - 23
Spanish - 345
Telugu - 77
Tamil - 81
Regards,
Bodhisattwa and Sai Phanindra
On Sun, 2 Feb 2025 at 09:49, Bodhisattwa bodhisattwa.rgkmc@gmail.com wrote:
Hello Wikisourcerers,
Thanks to Sai's commitment, we have successfully switched to a new reader this week and have resolved most of the rendering issues we were having with the last reader. The new reader is rendering the Wikisource stylings as they are. This new reader also comes with a wide range of other features to improve reader experience like search, navigation, highlight, bookmark, underline, annotation, font color options, font size options, font weight options, text normalisation, theme color options, image color inversion, page margin options, different font options, RTL support etc.
We have also included two more languages into the app as pilot i.e. Spanish and Marathi based on their recent contributions to the Wikidata items for book editions following the FRBR book model used in Wikidata:Books and app workflow. Special thanks to Ignacio, Subodh, Ashwini for the valuable contributions on Wikidata.
This is also a gentle reminder that the community conversation hour will happen today at 12:00 noon UTC as mentioned in the previous email.
See you all there,
Regards, Bodhisattwa
On Thu, 23 Jan 2025 at 12:19, Bodhisattwa bodhisattwa.rgkmc@gmail.com wrote:
Hello Wikisourcerers,
Thank you for all your feedback here, on meta-wiki and on telegram channels. We are keeping note of all of them and having discussions on them internally.
Our current e-book reader, which we are using in the app, has many limitations and does not render the stylings used in Wikisource properly. So, we are replacing it with another open-source ebook reader which has a variety of features and can overcome those limitations. It will take us a few more days to update the new code on Github. For this reason, we are postponing the conversation hour to 2nd February, same time.
- *Next community conversation hour*
- Sunday, 2 February 12:00 – 13:00 UTC
- Video call link: https://meet.google.com/khd-qvfy-nsr
Andy, we intend to give a demonstration of the app at the upcoming Wikisource conference and we have already requested for a small slot in the session to do that, but it is up to the conference program team to decide as the program submission deadline is already closed. Anyway, if it is not possible through official sessions, we can always demo at the meetups or outside official conference sessions.
Regards, Bodhisattwa
On Sat, 18 Jan 2025 at 07:32, Sam Wilson sam@samwilson.id.au wrote:
This app looks great! Very exciting.
On the topic of OPDS: there was some experimenting a few years ago with the OPDS feeds that ws-export currently produces, and the open source FBReader app which supports them: https://fbreader.org/book-sources
The main issues seemed to be around the actual navigation of the catalogue (people generally don't want to browse books alphabetically by title!). And that when OPDS feeds get too big they should ideally be broken up into subsidiary ones (e.g. by topic, for non-fiction, or genre for fiction). But those are not too hard to solve, and it does sound like building them from Wikidata data would be better (although I know there are disagreements about how much cataloguing to have on Wikidata vs Wikisource — that some Wikisources don't want topic, genre etc. to be decided by Wikidata).
There are various issues still with ws-export's stability, and reliability for exporting large or image-heavy books. I'm hoping to work on some of that this month (or rather, upgrade it to Symfony 7 first and then work on that stuff). I wonder if we should be looking again at caching the fully-generated epubs too (although that implies having fewer build-time options maybe).
On 18/1/25 01:22, thomas@pellissier-tanon.fr wrote:
[picked the wrong mailing list the first time, sorry for that]
This is amazing! Congrats to the team.
Magnus is raising a good point (thanks!), there is a standard API to
integrate catalog into epub reader apps. It's called OPDS [1] and some Wikisource(s) publishes one [2] since a decade. However, it's not very user friendly and we never made outreach to integrate it natively into apps so it's not much used. This OPDS feed is not based on Wikidata but on metadata present inside of Wikisource.
I still think having our own Wikisource branded app is relevant for
outreach and visibility, maybe building on OPDS instead of doing something bespoke would allow to reuse even more code?
Congrats again,
Thomas
[1] https://specs.opds.io/ [2] https://en.wikisource.org/wiki/Help:OPDS
Le vendredi 17 janvier 2025 à 15:37, Magnus Manske <
magnusmanske@googlemail.com> a écrit :
> This is great! > > Silly question: Considering that the reader app will display epub
format, is there no existing (open source) epub reader app that could use Wikisource as a, well, source? I would be surprised if there is not a single app where you can set/add an API URL, we would just have to be compatible with that API spec.
> > This could avoid a whole load of development and technical debt. > > On Fri, Jan 17, 2025 at 7:07 AM Bodhisattwa <
bodhisattwa.rgkmc@gmail.com> wrote:
> >> Hello Wikisourcerers, >> >> We are excited to announce that we have been developing a
Wikisource reader app for Android mobile users for the last three months and the beta version of the app is almost ready for testing. The development of the mobile app was a much-awaited request from the Wikisource community and when fully released, hopefully, it will help bring more readers to Wikisource platforms.
>> >> Necessary links of the reader app >> >> - A documentation page is in draft on meta wiki here -
https://meta.wikimedia.org/wiki/Wikisource_reader_app
>> - The Github repo of the code is here -
https://github.com/cis-india/Wikisource-Reader
>> >> Wikidata integration >> >> We have been using the FRBR data model used in the Wikidata
WikiProject Books to handle the descriptive metadata needed for filtering and categorizing the contents of the app. To be included in the app, an edition should have
>> >> - fully proofread or validated >> - transcluded into the main namespace or NS:0 >> - a Wikidata item with descriptive metadata property like title,
author etc.
>> - Wikisource index page url (P1957) property linked with that
item
>> - Wikisource sitelink for the NS:0 transcluded page linked with
that item
>> - a proofread and/or validated badge linked with that sitelink. >> >> Here are a few example of the book editions which are included in
the app
>> >> - An example of a Bangla book edition is this one >> - A list of all French books to be included in the app can be
found here - https://w.wiki/CkMP
>> >> Pilot languages and API >> >> In the beta version, we have piloted with English, French and
Bangla as these three languages are storing their metadata on Wikidata. We will roll out for other languages gradually. An API was built using Django and deployed on Toolforge which serves a catalogue of books following the data model described above. It periodically runs a set of SPARQL queries to retrieve data, processes that data and updates the database.
>> >> - Link - https://wsindex.toolforge.org/books/ >> - Repo - https://codeberg.org/ph4ni/wsindex >> >> Features >> >> We have decided to display the books in epub format which will be
generated through WS-export tool. Any issues related to epub generation can be reported here.
>> >> Right now, the following features are available >> >> 1. Browse through a list of works which are completely proofread
or validated.
>> 2. Filter based on the language or literary form of your choice. >> 3. Search for books by title or author. >> 4. Download books to your local device for offline access. >> 5. Share links to the Wikisource page of each book. >> 6. Read, delete, and track completion percentage of books in your
local library.
>> 7. Change font size, look up words, and access a dictionary. >> 8. Switch between dark and light themes for comfortable reading. >> 9. Use the app in multiple languages. >> >> Currently, we are working on the following areas >> >> - Internal epub reader which can render the styling of the
contents as much as possible
>> - Design of the contents and metadata within the app >> - Fixes here and there >> >> Next steps >> >> >> In coming days, we will >> >> - start conversation with the Wikisource communities to make
aware about the app, share the workflow needed to include contents there and get feedback for further development. (We had one call with the Marathi Wikisource community already).
>> >> - Next community conversation hour >> >> - Sunday, 26 January 12:00 – 13:00 UTC >> - Video call link: https://meet.google.com/khd-qvfy-nsr >> >> - discuss with all stakeholders regarding necessary steps. >> - register for Google Play Store and release the stable version
of the app there
>> - release the app in other play stores gradually. >> >> We would like to specially thank Sai Phanindra, who has developed
the app as an assignment from CIS-A2K.
>> >> Regards, >> Bodhisattwa >> (Part-time consultant, CIS-A2K) >> _______________________________________________ >> Wikisource-l mailing list -- wikisource-l@lists.wikimedia.org >> To unsubscribe send an email to
wikisource-l-leave@lists.wikimedia.org
Wikisource-l mailing list -- wikisource-l@lists.wikimedia.org To unsubscribe send an email to
wikisource-l-leave@lists.wikimedia.org _______________________________________________ Wikisource-l mailing list -- wikisource-l@lists.wikimedia.org To unsubscribe send an email to wikisource-l-leave@lists.wikimedia.org
Wikisource-l mailing list -- wikisource-l@lists.wikimedia.org To unsubscribe send an email to wikisource-l-leave@lists.wikimedia.org
wikisource-l@lists.wikimedia.org