Heya,
I have been talking with a lot of you in the past months and at Wikimania about Limn and how to move forward. One of the recurring themes has been that currently Limn is written in Coca and that significantly hinders adoption as there are very few Coco developers (Coco is a fork of Coffeescript).
I have sent this email to mobile-tech, e2 and e3 mailinglists as well because there are many developers outside of the Analytics team who use Limn and I would really like to hear their opinion as well.
So the question I want to pose is:
"Should we recompile Limn to either Coffeescript or Javascript or keep using Coco?"
This question is getting more urgent because of two reasons:
1) The Analytics team is going to grow in the coming months and we expect to start developing features for Limn again and if we want to drop Coco as dependency then this is probably the best time to talk about it.
2) It seems that the community around Coco is stagnant maybe even on the decline. When visiting https://github.com/satyr/coco you can see that there are very few commits in the last 4 months. This could either mean that the language is feature complete and bug free or more likely that the decline has started. For the long-term prospects of Limn, this is not good news.
I would like to run a strawpoll and please respond to this thread by answering with either Javascript, Coffeescript or Coco and optionally a short explanation.
Thanks! D
On Fri, Aug 9, 2013 at 4:18 PM, Diederik van Liere dvanliere@wikimedia.orgwrote:
I have been talking with a lot of you in the past months and at Wikimania about Limn and how to move forward. One of the recurring themes has been that currently Limn is written in Coca and that significantly hinders adoption as there are very few Coco developers (Coco is a fork of Coffeescript).
Limn needs two things to survive: users that like using it and developers that like developing it. You have the users. You need developers. Limn is not realistically going to attract substantial contributions from volunteers in the open source community, so you need to hire someone for the role. The job market for Coco developers is comparable in size and ontic status to the job market for tooth fairies, so you will need to hire a JavaScript developer, and you need to find a way to keep her or him motivated. The best way to do that is to tell this developer that you respect her intuitions and judgments and are trusting her to make the right call re: Coco vs. JavaScript and other matters related to software architecture. The specifications you provide should be expressed in terms of user requirements.
I would not attach too much significance to the choice of Coco vs. JavaScript. The choice of Coco drew so much ire not by dint of any feature of the language itself, but because the choice epitomized an approach to development that focused on self-gratification and posturing rather than simplicity and practicality. Dan's heroic efforts to clean it up have improved things considerably, but in my estimation Limn is not going to survive, because there are very few talented developers out there who are enticed by the prospect of cleaning up other people's mistakes as a full-time job.
It is important to realize that most of the functionality that users attribute to Limn is actually provided by external libraries (point in fact: vendor-bundle.min.js is 411k, limn.no-deps.min.js is 254k). I don't think that there is enough in that 254k to justify a substantial investment of resources.
In my opinion, the most sensible course forward is to feature-freeze Limn, hire a good, creative JavaScript developer with experience in data visualization, and commit 20% of this developer's time to Coco maintenance. The primarily purpose of this 20% time is to provide a context for the developer to interact with users and to understand what they like and what their wishes are. Anything beyond this 20% should decidedly not be encumbered by an indefensible a priori decision that Limn will be the visualization framework of the future.
Some quick background, important for what follows. Coco is a language that compiles to Javascript. It has tons of fancy features inspired by Perl and Ruby and Python. We don't use the vast majority of them in Limn. We use Coco because it's more productive than Javascript once you get comfortable with it. This took about two weeks for me and personally I think it was worth it. I'm happy to address specific points that others like Terry and Ori raised about Coco. Not all my answers will favor Coco but overall I like it. The second important thing is that Coffeescript is strictly worse than Coco. That's pretty easy to show and I point you to the satyr github page for examples.
Diederik, you estimate that the choice of language for Limn should be done before a new resource is hired to work on it and before we resume development of fixes/features. I am not so sure this is true. I think once we hire a talented Javascript developer and we decide as an organization what we want out of our visualization solution (read: not just Limn), we can make a much more informed decision. And there are other things to consider. For example Livescript is pretty popular. WebStorm *could* make Javascript development much more productive than it has been historically. Pure D3 might have become mature enough for us to rethink using it outright. And so on. I think at the moment we'd be giving an answer without a question.
Christian, as for historical reasons David used Coco. It's honestly mostly because it's a better more productive language and partly because David is a hardcore aesthete who seriously dislikes meaningless characters like brackets, etc.
Ori, I think your point about Limn not being the answer forever is great. We really do need to talk about what we want out of visualization and why we can't use the many, increasingly sophisticated frameworks being developed. Like pure D3 with a few templates or plugins via Rickshaw, Vega, etc. But, as to what's valuable in the non-library 250k or so of Limn code. The purpose of Limn was to make it fun to play with data in real time. We had to stop working on Limn before we could show progress towards that goal. The one unique thing in that 250k is the marriage of D3 with Knockout. This establishes a novel way of writing visualization components that are compatible with the end goal by design, out of the box. In other words, we now have only to write the UI via which people would actually play with the data. We know of no other work in this space and it's very exciting. It's the kind of stuff that Brett Victor was playing with in Tangle JS, and we would really further that kind of deep, intuitive understanding of data if we went forward.
So my suggestion is to figure out what we want first, and who's going to work on it. Then if we want pure Javascript, it would take a half hour to compile the Coco down and change some parts of the dev and build processes. On Aug 9, 2013 2:18 AM, "Ori Livneh" ori@wikimedia.org wrote:
On Fri, Aug 9, 2013 at 4:18 PM, Diederik van Liere < dvanliere@wikimedia.org> wrote:
I have been talking with a lot of you in the past months and at Wikimania about Limn and how to move forward. One of the recurring themes has been that currently Limn is written in Coca and that significantly hinders adoption as there are very few Coco developers (Coco is a fork of Coffeescript).
Limn needs two things to survive: users that like using it and developers that like developing it. You have the users. You need developers. Limn is not realistically going to attract substantial contributions from volunteers in the open source community, so you need to hire someone for the role. The job market for Coco developers is comparable in size and ontic status to the job market for tooth fairies, so you will need to hire a JavaScript developer, and you need to find a way to keep her or him motivated. The best way to do that is to tell this developer that you respect her intuitions and judgments and are trusting her to make the right call re: Coco vs. JavaScript and other matters related to software architecture. The specifications you provide should be expressed in terms of user requirements.
I would not attach too much significance to the choice of Coco vs. JavaScript. The choice of Coco drew so much ire not by dint of any feature of the language itself, but because the choice epitomized an approach to development that focused on self-gratification and posturing rather than simplicity and practicality. Dan's heroic efforts to clean it up have improved things considerably, but in my estimation Limn is not going to survive, because there are very few talented developers out there who are enticed by the prospect of cleaning up other people's mistakes as a full-time job.
It is important to realize that most of the functionality that users attribute to Limn is actually provided by external libraries (point in fact: vendor-bundle.min.js is 411k, limn.no-deps.min.js is 254k). I don't think that there is enough in that 254k to justify a substantial investment of resources.
In my opinion, the most sensible course forward is to feature-freeze Limn, hire a good, creative JavaScript developer with experience in data visualization, and commit 20% of this developer's time to Coco maintenance. The primarily purpose of this 20% time is to provide a context for the developer to interact with users and to understand what they like and what their wishes are. Anything beyond this 20% should decidedly not be encumbered by an indefensible a priori decision that Limn will be the visualization framework of the future.
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
From my perspective, the most important thing for the mobile department is
to have a tool that is easy to use, very easy for us to develop with, and generally requires the little overhead for us to work with and interact with the data we care about. I'd be surprised if anyone on the mobile team is particularly married to Limn. While I haven't interface with it much (aside from perusing graphs), it seems like Limn has been fairly trivial for us to use to generate our data visualizations, and as such has been great for us. Perhaps folks from the mobile dept who have spent development time with Limn ought to weigh in on this - Juliusz, Jon, Yuvi (anyone else)?
On Sat, Aug 10, 2013 at 7:45 AM, Dan Andreescu dandreescu@wikimedia.orgwrote:
Some quick background, important for what follows. Coco is a language that compiles to Javascript. It has tons of fancy features inspired by Perl and Ruby and Python. We don't use the vast majority of them in Limn. We use Coco because it's more productive than Javascript once you get comfortable with it. This took about two weeks for me and personally I think it was worth it. I'm happy to address specific points that others like Terry and Ori raised about Coco. Not all my answers will favor Coco but overall I like it. The second important thing is that Coffeescript is strictly worse than Coco. That's pretty easy to show and I point you to the satyr github page for examples.
Diederik, you estimate that the choice of language for Limn should be done before a new resource is hired to work on it and before we resume development of fixes/features. I am not so sure this is true. I think once we hire a talented Javascript developer and we decide as an organization what we want out of our visualization solution (read: not just Limn), we can make a much more informed decision. And there are other things to consider. For example Livescript is pretty popular. WebStorm *could* make Javascript development much more productive than it has been historically. Pure D3 might have become mature enough for us to rethink using it outright. And so on. I think at the moment we'd be giving an answer without a question.
Christian, as for historical reasons David used Coco. It's honestly mostly because it's a better more productive language and partly because David is a hardcore aesthete who seriously dislikes meaningless characters like brackets, etc.
Ori, I think your point about Limn not being the answer forever is great. We really do need to talk about what we want out of visualization and why we can't use the many, increasingly sophisticated frameworks being developed. Like pure D3 with a few templates or plugins via Rickshaw, Vega, etc. But, as to what's valuable in the non-library 250k or so of Limn code. The purpose of Limn was to make it fun to play with data in real time. We had to stop working on Limn before we could show progress towards that goal. The one unique thing in that 250k is the marriage of D3 with Knockout. This establishes a novel way of writing visualization components that are compatible with the end goal by design, out of the box. In other words, we now have only to write the UI via which people would actually play with the data. We know of no other work in this space and it's very exciting. It's the kind of stuff that Brett Victor was playing with in Tangle JS, and we would really further that kind of deep, intuitive understanding of data if we went forward.
So my suggestion is to figure out what we want first, and who's going to work on it. Then if we want pure Javascript, it would take a half hour to compile the Coco down and change some parts of the dev and build processes. On Aug 9, 2013 2:18 AM, "Ori Livneh" ori@wikimedia.org wrote:
On Fri, Aug 9, 2013 at 4:18 PM, Diederik van Liere < dvanliere@wikimedia.org> wrote:
I have been talking with a lot of you in the past months and at Wikimania about Limn and how to move forward. One of the recurring themes has been that currently Limn is written in Coca and that significantly hinders adoption as there are very few Coco developers (Coco is a fork of Coffeescript).
Limn needs two things to survive: users that like using it and developers that like developing it. You have the users. You need developers. Limn is not realistically going to attract substantial contributions from volunteers in the open source community, so you need to hire someone for the role. The job market for Coco developers is comparable in size and ontic status to the job market for tooth fairies, so you will need to hire a JavaScript developer, and you need to find a way to keep her or him motivated. The best way to do that is to tell this developer that you respect her intuitions and judgments and are trusting her to make the right call re: Coco vs. JavaScript and other matters related to software architecture. The specifications you provide should be expressed in terms of user requirements.
I would not attach too much significance to the choice of Coco vs. JavaScript. The choice of Coco drew so much ire not by dint of any feature of the language itself, but because the choice epitomized an approach to development that focused on self-gratification and posturing rather than simplicity and practicality. Dan's heroic efforts to clean it up have improved things considerably, but in my estimation Limn is not going to survive, because there are very few talented developers out there who are enticed by the prospect of cleaning up other people's mistakes as a full-time job.
It is important to realize that most of the functionality that users attribute to Limn is actually provided by external libraries (point in fact: vendor-bundle.min.js is 411k, limn.no-deps.min.js is 254k). I don't think that there is enough in that 254k to justify a substantial investment of resources.
In my opinion, the most sensible course forward is to feature-freeze Limn, hire a good, creative JavaScript developer with experience in data visualization, and commit 20% of this developer's time to Coco maintenance. The primarily purpose of this 20% time is to provide a context for the developer to interact with users and to understand what they like and what their wishes are. Anything beyond this 20% should decidedly not be encumbered by an indefensible a priori decision that Limn will be the visualization framework of the future.
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Just a quick note to respond to Arthur - when I was setting up mobile-reportcard, limn was usable only because Dan was being supremely awesome and hand-holding me through a lot of stuff. There is absolutely no documentation, and I can't just read the code because I'm not familiar with Coco (and considering its stated goal to be closer to perl + lack of a community, it is not something I want to spend time on). Having it in Coco also means I can't really fix things when I run into bugs. Having to hand-write the config files was very painful, considering that it was JSON and there wasn't really any documentation. I later heard that there's YAML available, but that I did not know at that time because... there was no documentation. I personally found Limn a lot more painful than I thought it needed to be, with a bit of too-much-flexibility and over-configuration issues - everything seems possible but nothing seems easy. Adding a simple time series chart should not require as much fiddling as it requires now. If I were building a dashboard for personal use, Limn would definitely be off the cards.
Diedrik also started another thread about what Limn should be and should not be, and I think that is a much more important thing than language choice. I've still never understood if it was supposed to be a 'here is some data, make some nice graphs so we can track key metrics!' software for building reportcards and dashboards? Or is it a super-flexible data-warehouse kinda thing where you can explore data and make charts to figure out what is going on (aka excel on steroids)? I personally find that it tries to be the latter but we are using it to be the former and that sortof sucks. Having clarity on what the product should be will alleviate a lot of my pain points, I think.
I belief that the consensus for now is to leave Coco where it is and first reduce the complexity of the codebase itself. Decompiling it to Javascript is less of a task then I assumed it is so we can always do that but Dan assured me that Coco is not the obstacle. @Yuvi: if you have a strong desire to submit a patch and Coco is putting you off then please reach out to us on IRC and we can give you a 10 minute crash course :) D
On Sun, Sep 1, 2013 at 8:19 PM, Yuvi Panda yuvipanda@gmail.com wrote:
*bump*
Did something actionable come out of this?
-- Yuvi Panda T http://yuvi.in/blog
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Hi,
On Fri, Aug 09, 2013 at 10:18:12AM +0200, Diederik van Liere wrote:
I would like to run a strawpoll and please respond to this thread by answering with either Javascript, Coffeescript or Coco [...]
Coco comes with some syntactic sugar, but since Coco is not one of the standard languages I assume the choice for Coco back then means that Coco comes with real killer features that make Coco like a perfect match for limn.
We would have to make up for those features when switching to other languages.
However, doing a quick search for the Coco choice on limn, I could not find any such discussion.
Is there some writeup on why Coco has been chosen, or which aspects of Coco/limn make them a perfect match?
Best regards, Christian
I think the question of paid developer time is somewhat orthogonal to that of language choice, save for one element. In general you don't want to be the biggest user of a given language/framework/what-have/you (I'm just going to say framework to keep it general) unless you're willing to upstream patches to that framework and forego the advantages of broad use. Sometimes that's valuable if the framework is particularly interesting but in almost all cases you'll want a paid developer working on the project in a serious capacity.
However if the project depends on more common and robust frameworks then the issue of a paid developer is less of a problem. If we were choosing between coffeescript and JavaScript, it wouldn't really come up.
As for the language choice, I'd recommend JS but the project is large enough that rebuilding it in a different language may be too much.
-Adam
On Fri, Aug 9, 2013 at 9:45 AM, Christian Aistleitner < christian@quelltextlich.at> wrote:
Hi,
On Fri, Aug 09, 2013 at 10:18:12AM +0200, Diederik van Liere wrote:
I would like to run a strawpoll and please respond to this thread by answering with either Javascript, Coffeescript or Coco [...]
Coco comes with some syntactic sugar, but since Coco is not one of the standard languages I assume the choice for Coco back then means that Coco comes with real killer features that make Coco like a perfect match for limn.
We would have to make up for those features when switching to other languages.
However, doing a quick search for the Coco choice on limn, I could not find any such discussion.
Is there some writeup on why Coco has been chosen, or which aspects of Coco/limn make them a perfect match?
Best regards, Christian
-- ---- quelltextlich e.U. ---- \ ---- Christian Aistleitner ---- Companies' registry: 360296y in Linz Christian Aistleitner Gruendbergstrasze 65a Email: christian@quelltextlich.at 4040 Linz, Austria Phone: +43 732 / 26 95 63 Fax: +43 732 / 26 95 63 Homepage: http://quelltextlich.at/
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics