Hi
Its final evaluation period, so I take this time to write some final notes on the project I worked on this summer, that is integrating collaborative editing into the VisualEditor project[1]. So to fit rightly into the scope of GSoC, the whole collaborative thing on VisualEditor was split into two phases, of which, the first phase was aimed to be delivered. The system is to allow only one user to make changes to a document in an editing session, while the others can spectate the changes in realtime, therefore halting at a stage where concurrency issues arising from multiple editors, would be deal with.
The project is majorly composed of the server-side and client-side components. A server(collaboration server in this context) has been built, which co-ordinates different clients participating in an editing session. Some of the things that the collaboration server does are -
1. Maintains a centralized copy(server's copy) of the document that is worked on in an editing session. 2. Listens to the incoming transactions from a client who makes a new change to the document and applies those transactions to the server's copy of the document. 3. Broadcasts an incoming transaction to the other clients subscribed to the same editing session that triggered the transaction, so that they can apply that transaction to their local documents and keep in sync. 4. Re-use the VisualEditor modules to create a binding for taking care of the above. 5. Talks to an external parsoid service, when it needs the HTML feeding for initializing a document. Initially it used parsoid modules to parse wiki content internally, but considering the scalability, this is done over HTTP using an external parsoid service. 6. And ofcourse, it scales the above features for multiple editing sessions.
The server is built in node.js and uses socket.io library for making persistent connections with the clients. For testing, nodeunit has been used.
The other major aspect of the project is, the client module, which hooks into VisualEditor, initiates a connection with the server, and does all the things with transactions including translating them to the server. The components of client module, like other components of VE, interact using events, which has allowed me to test them easily. And, for testing some major parts of the code, qunit has been used.
So the above, makes for more than 90% of all that I wanted to do. Some of the things that remain are -
1. Authentication of users on collaboration server - Currently, the users which connect to the collaboration server are not validated before putting them on one of the editing sessions. To overcome this, there has to be way by which MediaWiki could help in validating a user coupled with some extra information by a third party. This called for the need of OAuth, but since OAuth isn't ready yet, Roan and Trevor suggested to have a new API module for doing that. I've implemented the API module[2], and written the authentication part in my project aligned with the implementation. So its nearly done.
2. Transfer of editing control - This is one feature I've not worked on due to confusions mainly related to correctly attributing edit credits to particular users which took part in the editing session. I think, this should not be hard to implement, once the confusions are stripped away.
The above are part of the tasks in my current plan. My plan for future is to further refine the code that I've written so far, especially the server side code. There have been umpteen times when I've shown my demo to people, and they've said, 'dude, it would be amazing if it supported multiple editors', that has always enticed me to work on supporting multiple editors. But, I realized, it was really good decision to cut the scope till what has been done for the sake of robustness. Although, it will be a challenging task, but its definitely in my plan to work on supporting multiple editors in future. The time when we won't need Etherpad, hopefully :)
It has been a great fun and learning time working along side some bright students, dedicated volunteers and amazing Wikimedia engineers, especially the VisualEditor team. Special thanks to Trevor, Gabriel, Roan, Mark(traceur), and all those who ever showed interest in my project. Definitely looking forward to more awesome work.
Cheers!
Apologize for forgetting to put in the links there.
[1] - http://www.mediawiki.org/wiki/User:Dash1291/GSoC_2012_Application [2] - https://gerrit.wikimedia.org/r/20013
On Wed, Aug 22, 2012 at 9:02 PM, Ashish Dubey ashish.dubey91@gmail.comwrote:
Hi
Its final evaluation period, so I take this time to write some final notes on the project I worked on this summer, that is integrating collaborative editing into the VisualEditor project[1]. So to fit rightly into the scope of GSoC, the whole collaborative thing on VisualEditor was split into two phases, of which, the first phase was aimed to be delivered. The system is to allow only one user to make changes to a document in an editing session, while the others can spectate the changes in realtime, therefore halting at a stage where concurrency issues arising from multiple editors, would be deal with.
The project is majorly composed of the server-side and client-side components. A server(collaboration server in this context) has been built, which co-ordinates different clients participating in an editing session. Some of the things that the collaboration server does are -
- Maintains a centralized copy(server's copy) of the document that is
worked on in an editing session. 2. Listens to the incoming transactions from a client who makes a new change to the document and applies those transactions to the server's copy of the document. 3. Broadcasts an incoming transaction to the other clients subscribed to the same editing session that triggered the transaction, so that they can apply that transaction to their local documents and keep in sync. 4. Re-use the VisualEditor modules to create a binding for taking care of the above. 5. Talks to an external parsoid service, when it needs the HTML feeding for initializing a document. Initially it used parsoid modules to parse wiki content internally, but considering the scalability, this is done over HTTP using an external parsoid service. 6. And ofcourse, it scales the above features for multiple editing sessions.
The server is built in node.js and uses socket.io library for making persistent connections with the clients. For testing, nodeunit has been used.
The other major aspect of the project is, the client module, which hooks into VisualEditor, initiates a connection with the server, and does all the things with transactions including translating them to the server. The components of client module, like other components of VE, interact using events, which has allowed me to test them easily. And, for testing some major parts of the code, qunit has been used.
So the above, makes for more than 90% of all that I wanted to do. Some of the things that remain are -
- Authentication of users on collaboration server - Currently, the users
which connect to the collaboration server are not validated before putting them on one of the editing sessions. To overcome this, there has to be way by which MediaWiki could help in validating a user coupled with some extra information by a third party. This called for the need of OAuth, but since OAuth isn't ready yet, Roan and Trevor suggested to have a new API module for doing that. I've implemented the API module[2], and written the authentication part in my project aligned with the implementation. So its nearly done.
- Transfer of editing control - This is one feature I've not worked on
due to confusions mainly related to correctly attributing edit credits to particular users which took part in the editing session. I think, this should not be hard to implement, once the confusions are stripped away.
The above are part of the tasks in my current plan. My plan for future is to further refine the code that I've written so far, especially the server side code. There have been umpteen times when I've shown my demo to people, and they've said, 'dude, it would be amazing if it supported multiple editors', that has always enticed me to work on supporting multiple editors. But, I realized, it was really good decision to cut the scope till what has been done for the sake of robustness. Although, it will be a challenging task, but its definitely in my plan to work on supporting multiple editors in future. The time when we won't need Etherpad, hopefully :)
It has been a great fun and learning time working along side some bright students, dedicated volunteers and amazing Wikimedia engineers, especially the VisualEditor team. Special thanks to Trevor, Gabriel, Roan, Mark(traceur), and all those who ever showed interest in my project. Definitely looking forward to more awesome work.
Cheers!
-- Ashish Dubey
On 22/08/12 17:32, Ashish Dubey wrote:
The above are part of the tasks in my current plan. My plan for future is to further refine the code that I've written so far, especially the server side code. There have been umpteen times when I've shown my demo to people, and they've said, 'dude, it would be amazing if it supported multiple editors', that has always enticed me to work on supporting multiple editors. But, I realized, it was really good decision to cut the scope till what has been done for the sake of robustness. Although, it will be a challenging task, but its definitely in my plan to work on supporting multiple editors in future. The time when we won't need Etherpad, hopefully :)
People will always be asking more. What you made is already amazing. Is there a public testing server somewhere? If not, it may be worth making a public demostration with people watching you edit (it may also double as load testing).
Best regards
The previous demo[1] that I set up is broken and anyways old. I'll soon post the link to the new demo I'm setting up.
[1] - http://ashishdubey.info/mw/index.php/Main_Page
On Thu, Aug 23, 2012 at 3:50 AM, Platonides Platonides@gmail.com wrote:
On 22/08/12 17:32, Ashish Dubey wrote:
The above are part of the tasks in my current plan. My plan for future is to further refine the code that I've written so far, especially the
server
side code. There have been umpteen times when I've shown my demo to
people,
and they've said, 'dude, it would be amazing if it supported multiple editors', that has always enticed me to work on supporting multiple editors. But, I realized, it was really good decision to cut the scope
till
what has been done for the sake of robustness. Although, it will be a challenging task, but its definitely in my plan to work on supporting multiple editors in future. The time when we won't need Etherpad,
hopefully
:)
People will always be asking more. What you made is already amazing. Is there a public testing server somewhere? If not, it may be worth making a public demostration with people watching you edit (it may also double as load testing).
Best regards
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
wikitech-l@lists.wikimedia.org