I couldn't find a tool to convert my videos from whatever format into .ogv outside my PC box before pushing to Commons. I guess there might exist something like that, but perhaps I can't find it. But I made one for myself. Maybe it might be useful to others too.
I call it CommonsConvert. Upload any video format, enter username and password, and get the video converted into .ogv format plus pushed to Commons all in one shot. You upload the original video in whatever format, and get it on Commons in .ogv
Some rough edges currently, such as can't/don't know/no available means to append license to uploaded video among other issues. Working on the ones I can asap.
It uses mwclient module via Django based on Python. Django gives user info to Python, Python calls avconv in a subprocess, and the converted file is relayed to Commons via mwclient module via Media Wiki API.
I think not everyone has means/technical know how/interest/time converting videos taken on their PC to an Ogg-Vorbis-whatever format before uploading to Commons.
Doing the conversion on a server leaves room for user to focus on getting videos than processing them.
I don't know if this is or will be of any interest that someone might wanna use, but I personally would enjoy having a server sitting somewhere convert my videos I want to save onto Commons, than using my local computer doing that boring task.
In an email to this list a week or so ago, I 'ranted' about why commons wants a specific format (which if not for commons, I never come across that format anywhere), but has no provision for converting any videos thrown at it into that format of its choice. Well....
Tool can be found here: khophi.co http://khophi.co/commonsconvert/ http://khophi.co/commonsconvertcommonsconvert http://khophi.co/commonsconvert
And this is sample video uploaded using the tool. https://commons.m.wikimedia.org/wiki/File:Testing_file_for_upload_last.ogv (will be deleted soon, likely)
What I do not know or have not experimented yet is whether uploading using the api also has the 100mb upload restriction.
Will appreciate feedback.
Very nice! However, according to https://meta.wikimedia.org/wiki/Terms_of_use, users' passwords should never be disclosed to any third party. Please read OAuth/For Developers https://www.mediawiki.org/wiki/OAuth/For_Developers and convert your tool to use OAuth instead ;-)
Il 04/02/2015 01:43, Nkansah Rexford ha scritto:
I couldn't find a tool to convert my videos from whatever format into .ogv outside my PC box before pushing to Commons. I guess there might exist something like that, but perhaps I can't find it. But I made one for myself. Maybe it might be useful to others too.
I call it CommonsConvert. Upload any video format, enter username and password, and get the video converted into .ogv format plus pushed to Commons all in one shot. You upload the original video in whatever format, and get it on Commons in .ogv
Some rough edges currently, such as can't/don't know/no available means to append license to uploaded video among other issues. Working on the ones I can asap.
It uses mwclient module via Django based on Python. Django gives user info to Python, Python calls avconv in a subprocess, and the converted file is relayed to Commons via mwclient module via Media Wiki API.
I think not everyone has means/technical know how/interest/time converting videos taken on their PC to an Ogg-Vorbis-whatever format before uploading to Commons.
Doing the conversion on a server leaves room for user to focus on getting videos than processing them.
I don't know if this is or will be of any interest that someone might wanna use, but I personally would enjoy having a server sitting somewhere convert my videos I want to save onto Commons, than using my local computer doing that boring task.
In an email to this list a week or so ago, I 'ranted' about why commons wants a specific format (which if not for commons, I never come across that format anywhere), but has no provision for converting any videos thrown at it into that format of its choice. Well....
Tool can be found here: khophi.co http://khophi.co/commonsconvert/ http://khophi.co/commonsconvertcommonsconvert http://khophi.co/commonsconvert
And this is sample video uploaded using the tool. https://commons.m.wikimedia.org/wiki/File:Testing_file_for_upload_last.ogv (will be deleted soon, likely)
What I do not know or have not experimented yet is whether uploading using the api also has the 100mb upload restriction.
Will appreciate feedback. _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Feb 3, 2015 8:43 PM, "Nkansah Rexford" seanmavley@gmail.com wrote:
I couldn't find a tool to convert my videos from whatever format into .ogv outside my PC box before pushing to Commons. I guess there might exist something like that, but perhaps I can't find it. But I made one for myself. Maybe it might be useful to others too.
I call it CommonsConvert. Upload any video format, enter username and password, and get the video converted into .ogv format plus pushed to Commons all in one shot. You upload the original video in whatever format, and get it on Commons in .ogv
Some rough edges currently, such as can't/don't know/no available means to append license to uploaded video among other issues. Working on the ones I can asap.
It uses mwclient module via Django based on Python. Django gives user info to Python, Python calls avconv in a subprocess, and the converted file is relayed to Commons via mwclient module via Media Wiki API.
I think not everyone has means/technical know how/interest/time converting videos taken on their PC to an Ogg-Vorbis-whatever format before
uploading
to Commons.
Doing the conversion on a server leaves room for user to focus on getting videos than processing them.
I don't know if this is or will be of any interest that someone might
wanna
use, but I personally would enjoy having a server sitting somewhere
convert
my videos I want to save onto Commons, than using my local computer doing that boring task.
In an email to this list a week or so ago, I 'ranted' about why commons wants a specific format (which if not for commons, I never come across
that
format anywhere), but has no provision for converting any videos thrown at it into that format of its choice. Well....
Tool can be found here: khophi.co http://khophi.co/commonsconvert/ http://khophi.co/commonsconvertcommonsconvert http://khophi.co/commonsconvert
And this is sample video uploaded using the tool. https://commons.m.wikimedia.org/wiki/File:Testing_file_for_upload_last.ogv (will be deleted soon, likely)
What I do not know or have not experimented yet is whether uploading using the api also has the 100mb upload restriction.
Will appreciate feedback. _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Cool. Thanks for working on this sort of thing. The uploading videos process certainly benefit from some love.
May i suggest using webm (of the vorbis/vp8 variety) as the output format? Webm will give higher quality videos at lower file size, so is probably a better choice if converting from another format.
For the 100mb thing - there are multiple ways to upload things with the api. The chunked method has a file size limit of 1gb. All the other methods have the 100mb limit.
If you havent already, id encourage mentioning this on [[Commons:VP]]. "Real" users would probably be able to give much more specific feedback.
Cheers, Bawolff
P.s. im not sure, but i think user:Prolineserver might have been working on something similar, in case you are looking for collaborators.
On Tue, Feb 3, 2015 at 5:19 PM, Brian Wolff bawolff@gmail.com wrote:
On Feb 3, 2015 8:43 PM, "Nkansah Rexford" seanmavley@gmail.com wrote:
I couldn't find a tool to convert my videos from whatever format into .ogv outside my PC box before pushing to Commons. I guess there might exist something like that, but perhaps I can't find it. But I made one for myself. Maybe it might be useful to others too.
I call it CommonsConvert. Upload any video format, enter username and password, and get the video converted into .ogv format plus pushed to Commons all in one shot. You upload the original video in whatever format, and get it on Commons in .ogv
Some rough edges currently, such as can't/don't know/no available means to append license to uploaded video among other issues. Working on the ones I can asap.
It uses mwclient module via Django based on Python. Django gives user info to Python, Python calls avconv in a subprocess, and the converted file is relayed to Commons via mwclient module via Media Wiki API.
I think not everyone has means/technical know how/interest/time converting videos taken on their PC to an Ogg-Vorbis-whatever format before
uploading
to Commons.
Doing the conversion on a server leaves room for user to focus on getting videos than processing them.
I don't know if this is or will be of any interest that someone might
wanna
use, but I personally would enjoy having a server sitting somewhere
convert
my videos I want to save onto Commons, than using my local computer doing that boring task.
In an email to this list a week or so ago, I 'ranted' about why commons wants a specific format (which if not for commons, I never come across
that
format anywhere), but has no provision for converting any videos thrown at it into that format of its choice. Well....
Tool can be found here: khophi.co http://khophi.co/commonsconvert/ http://khophi.co/commonsconvertcommonsconvert http://khophi.co/commonsconvert
And this is sample video uploaded using the tool. https://commons.m.wikimedia.org/wiki/File:Testing_file_for_upload_last.ogv (will be deleted soon, likely)
What I do not know or have not experimented yet is whether uploading using the api also has the 100mb upload restriction.
Will appreciate feedback. _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Cool. Thanks for working on this sort of thing. The uploading videos process certainly benefit from some love.
May i suggest using webm (of the vorbis/vp8 variety) as the output format? Webm will give higher quality videos at lower file size, so is probably a better choice if converting from another format.
For the 100mb thing - there are multiple ways to upload things with the api. The chunked method has a file size limit of 1gb. All the other methods have the 100mb limit.
See https://commons.wikimedia.org/wiki/Commons:Chunked_uploads for more information.
If you havent already, id encourage mentioning this on [[Commons:VP]]. "Real" users would probably be able to give much more specific feedback.
Cheers, Bawolff
P.s. im not sure, but i think user:Prolineserver might have been working on something similar, in case you are looking for collaborators.
That's https://tools.wmflabs.org/videoconvert/ - seems to be down right now, but usually it works very well for converting to WebM, and can also upload the result directly to Commons using OAuth. It's just not very fast (can take several hours to generate a 200MB WebM).
Results: https://commons.wikimedia.org/wiki/Category:Uploaded_with_videoconvert
Hi,
there is the tool by Holger which does that using WebM - but it is hosted on WMF Labs. It uses OAuth for unified login and moving the files to Commons.
Then, a more elaborate tool for * storing raw material at the Internet Archive * generating patent free WebM proxy clips for editing * rendering high-quality videos * moving these rendered videos to Commons directly
is the Video Editing Server, developed by some Wikipedians and a MLT developer, hosted by the Internet Archive:
https://wikimedia.meltvideo.com/
It also uses OAuth for login and moving files to Commons.
The workflow with this:
# upload all your raw files to the server for ## long-term storage ## to make them available to other editors ## to let the server use them in the rendering process
# the server transcodes all files into WebM "proxy clips"
# editors download the WebM proxy clips ## do the editing on your computer ## create an MLT project file (eg. using kdenlive or another MLT-based video editor)
# upload the project file ## server will replace proxy clips with raw material ## server will render video project ## server will move generated file to Commons
It comes with a search engine, meta data forms... it's still pretty new (development started in December '14) but can be used. We plan to add some more features like tagging using Wikidata QIDs (hence allowing multilingual / localised tagging / searching, adding more project file formats and renderer, making old project file revisions available for download, give it a nice vector-based theme, give it a better domain name and SSL certificate...
Play with it and have fun!
For source code or any issues refer to GitHub: https://github.com/ddennedy/wikimedia-video-editing-server
also there see the wiki for the specs and a deployment guide: https://github.com/ddennedy/wikimedia-video-editing-server/wiki
/Manuel
Thanks for all the ideas and suggestions. Will start with enabling oauth right away.
I'm dead in php so I always look for python related hooks. I think this will do for the oauth? http://pythonhosted.org/mwoauth/
As regards the conversion to webm instead of ogv, I tried but couldn't get avconv to process files into the webm format. Perhaps I'm not putting things together right. Will read more on that.
Its very interesting for me to know similar tools exist in the system. I wish a tool like https://tools.wmflabs.org/videoconvert/ was listed on the ways to upload videos on commons page. I would have used it right away.
Really appreciate all your thoughts. With MediaWiki I wish I was a bit good in php. Thus I always look around for python related packages to interact with the API. But with the oauth module found, I think authentication won't be the 'crude' way I'm doing it now.
Will reach out to 'real' users on VP soon too. Hi,
there is the tool by Holger which does that using WebM - but it is hosted on WMF Labs. It uses OAuth for unified login and moving the files to Commons.
Then, a more elaborate tool for * storing raw material at the Internet Archive * generating patent free WebM proxy clips for editing * rendering high-quality videos * moving these rendered videos to Commons directly
is the Video Editing Server, developed by some Wikipedians and a MLT developer, hosted by the Internet Archive:
https://wikimedia.meltvideo.com/
It also uses OAuth for login and moving files to Commons.
The workflow with this:
# upload all your raw files to the server for ## long-term storage ## to make them available to other editors ## to let the server use them in the rendering process
# the server transcodes all files into WebM "proxy clips"
# editors download the WebM proxy clips ## do the editing on your computer ## create an MLT project file (eg. using kdenlive or another MLT-based video editor)
# upload the project file ## server will replace proxy clips with raw material ## server will render video project ## server will move generated file to Commons
It comes with a search engine, meta data forms... it's still pretty new (development started in December '14) but can be used. We plan to add some more features like tagging using Wikidata QIDs (hence allowing multilingual / localised tagging / searching, adding more project file formats and renderer, making old project file revisions available for download, give it a nice vector-based theme, give it a better domain name and SSL certificate...
Play with it and have fun!
For source code or any issues refer to GitHub: https://github.com/ddennedy/wikimedia-video-editing-server
also there see the wiki for the specs and a deployment guide: https://github.com/ddennedy/wikimedia-video-editing-server/wiki
/Manuel -- Wikimedia CH - Verein zur Förderung Freien Wissens Lausanne, +41 (21) 34066-22 - www.wikimedia.ch
_______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I wrote a PHP class to specifically deal with the Wikimedia OAuth, including uploads (not chunked though). May be helpful.
https://bitbucket.org/magnusmanske/magnustools/src/9dc80c2479a41239b9661b355...
On Wed Feb 04 2015 at 09:29:28 Nkansah Rexford seanmavley@gmail.com wrote:
Thanks for all the ideas and suggestions. Will start with enabling oauth right away.
I'm dead in php so I always look for python related hooks. I think this will do for the oauth? http://pythonhosted.org/mwoauth/
As regards the conversion to webm instead of ogv, I tried but couldn't get avconv to process files into the webm format. Perhaps I'm not putting things together right. Will read more on that.
Its very interesting for me to know similar tools exist in the system. I wish a tool like https://tools.wmflabs.org/videoconvert/ was listed on the ways to upload videos on commons page. I would have used it right away.
Really appreciate all your thoughts. With MediaWiki I wish I was a bit good in php. Thus I always look around for python related packages to interact with the API. But with the oauth module found, I think authentication won't be the 'crude' way I'm doing it now.
Will reach out to 'real' users on VP soon too. Hi,
there is the tool by Holger which does that using WebM - but it is hosted on WMF Labs. It uses OAuth for unified login and moving the files to Commons.
Then, a more elaborate tool for
- storing raw material at the Internet Archive
- generating patent free WebM proxy clips for editing
- rendering high-quality videos
- moving these rendered videos to Commons directly
is the Video Editing Server, developed by some Wikipedians and a MLT developer, hosted by the Internet Archive:
https://wikimedia.meltvideo.com/
It also uses OAuth for login and moving files to Commons.
The workflow with this:
# upload all your raw files to the server for ## long-term storage ## to make them available to other editors ## to let the server use them in the rendering process
# the server transcodes all files into WebM "proxy clips"
# editors download the WebM proxy clips ## do the editing on your computer ## create an MLT project file (eg. using kdenlive or another MLT-based video editor)
# upload the project file ## server will replace proxy clips with raw material ## server will render video project ## server will move generated file to Commons
It comes with a search engine, meta data forms... it's still pretty new (development started in December '14) but can be used. We plan to add some more features like tagging using Wikidata QIDs (hence allowing multilingual / localised tagging / searching, adding more project file formats and renderer, making old project file revisions available for download, give it a nice vector-based theme, give it a better domain name and SSL certificate...
Play with it and have fun!
For source code or any issues refer to GitHub: https://github.com/ddennedy/wikimedia-video-editing-server
also there see the wiki for the specs and a deployment guide: https://github.com/ddennedy/wikimedia-video-editing-server/wiki
/Manuel
Wikimedia CH - Verein zur Förderung Freien Wissens Lausanne, +41 (21) 34066-22 - www.wikimedia.ch
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hi Magnus,
On 02/04/2015 04:40 PM, Magnus Manske wrote:
I wrote a PHP class to specifically deal with the Wikimedia OAuth, including uploads (not chunked though). May be helpful.
https://bitbucket.org/magnusmanske/magnustools/src/9dc80c2479a41239b9661b355...
very good! Because I have written a PHP library to use the MW library - it does not OAuth, because it is command-line based, but it can do chunked uploads!
Maybe there is a way to add OAuth to it, then I will shamelessly steal from your code.
My code is here:
https://github.com/masterssystems/phpapibot
I know that at least Dan used some it for inspiration for the video editing server upload component ;-)
/Manuel
On Wed, Feb 4, 2015 at 1:28 AM, Nkansah Rexford seanmavley@gmail.com wrote:
Its very interesting for me to know similar tools exist in the system. I wish a tool like https://tools.wmflabs.org/videoconvert/ was listed on the ways to upload videos on commons page. I would have used it right away.
I added a section "Online conversion tools" to https://commons.wikimedia.org/wiki/Help:Converting_video
-- =S Page WMF Tech writer
Thank you for adding that. I'm sure when my tool is ready using OAuth, it might be added to the list.
Is it possible to run/host a python based application on wmflabs? On Feb 6, 2015 7:25 PM, "S Page" spage@wikimedia.org wrote:
On Wed, Feb 4, 2015 at 1:28 AM, Nkansah Rexford seanmavley@gmail.com wrote:
Its very interesting for me to know similar tools exist in the system. I wish a tool like https://tools.wmflabs.org/videoconvert/ was listed on
the
ways to upload videos on commons page. I would have used it right away.
I added a section "Online conversion tools" to https://commons.wikimedia.org/wiki/Help:Converting_video
-- =S Page WMF Tech writer _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Fri, Feb 6, 2015 at 12:07 PM, Nkansah Rexford nkansahrexford@gmail.com wrote:
Is it possible to run/host a python based application on wmflabs?
Sure, there are a bunch there. https://wikitech.wikimedia.org/wiki/Help:Tool_Labs#Quick_start (get a labs account, create a new tools account or maybe join the existing videoconvert tool account).
wikitech-l@lists.wikimedia.org