There is a remaining bug. Try the following e.g. from idle:
import Wikipedia
you are asked to install patch.exe. If you decline this request you are asked a second time. If declined a second time the script raises a fatal error while importing beautifullsoup. Beautifullsoup is a central part if the screen scraping compat framework and we should not prohibit using the framework for a cosmetic patch. Please give us back that part as e.g. externals or a library script. Anyway it is not a good idea to ask for installation (twice) and fall into a fatal error if you answer is NO. There should be alternatives to patching the working copy (e.g. nightly dump etc.)
Greetings xqt
----- Ursprüngliche Nachricht ----- Von: Dr. Trigon Gesendet: 03.07.2013 20:14 An: pywikipedia-l@lists.wikimedia.org Betreff: Re: [Pywikipedia-l] Question about externals and patches
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hello to all (again)!
Quit some time has elapsed since my first mail. Today I tested successfully 'patch.exe' [1] in win and in linux wine. This is another one than mentioned before, but/and as I can see works quite well even with LF (unix-style) line endings.
[1] http://gnuwin32.sourceforge.net/packages/patch.htm
Since I will commit these changes now the time has come to test it on other machines also. After that was successful too I will remove BeautifulSoup.py from externals because it will be downloaded and patched automatically.
In order to test this on your windows machine you can e.g.: 1.) create a directory, e.g. '[...]/Desktop/patch-test' 2.) download and extract 'patch.exe' from http://downloads.sourceforge.net/project/gnuwin32/patch/2.5.9-7/patch-2.5.9-... to this directory 3.) download and extract 'BeautifulSoup.py' from https://pypi.python.org/packages/source/B/BeautifulSoup/BeautifulSoup-3.2.0.... to this directory 4.) download the patch 'patch-BeautifulSoup' from http://svn.wikimedia.org/viewvc/pywikipedia/trunk/pywikipedia/externals/patc... to this directory 5.) open 'cmd', cd to '[...]/Desktop/patch-test' and enter "patch.exe - -p0 -d [...]/Desktop/patch-test/ -i [...]/Desktop/patch-test/patch-BeautifulSoup --binary" 6.) check the output for error messages and compare the resulting 'BeautifulSoup.py' to the one in your pywikipedia/externals directory, they should be exactly the same! IF NOT AND/OR ERRORS OCCURED PLEASE REPORT THIS TO ME!
Thanks a lot to everybody helping and Greetings DrTrigon
On 25.05.2013 11:40, Dr. Trigon wrote:
Hello all!
Following issue: I am up re-organizing the whole "externals" part in trunk as you might have recognized already. In fact this is done now with the single exception of a generic patching system, e.g. needed for BeautifulSoup.py. (As usual) this is no problem under linux, but becomes a major issue under win.
The mechanism I want to use is the well known diff-patch duo. Therefore a "patch" executable/binary (OR python script) is needed (for every OS). While this is kind of "built-in" in linux, win needs extra-attention. This is what I found so far:
- http://sourceforge.net/projects/unxutils/ The executables do only
depend on the Microsoft C-runtime (msvcrt.dll) and not on an emulation layer like that provided by Cygwin tools - windows ONLY not multi OS
script therefore multi OS - but does not support the full diff-patch "command set" e.g. cannot create new files
command-line tool like "patch" but a python library/module. Multi OS.
So I am stuck here and need some further knowledge, experience and personal preferences from your side in order to make a good decicion.
In my opinion we should also keep further os (than just linux, mac, win) in our mind, becuase they are very close to what we already have too.
Thanks for any help and Greetings DrTrigon
_______________________________________________ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
_______________________________________________ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 01.08.2013 05:37, info@gno.de wrote:
There is a remaining bug. Try the following e.g. from idle:
import Wikipedia
you are asked to install patch.exe. If you decline this request you are asked a second time. If declined a second time the script raises a fatal error while importing beautifullsoup. Beautifullsoup is a central part if the screen scraping compat framework and we should not prohibit using the framework for a cosmetic patch. Please give us back that part as e.g. externals or a library script. Anyway it is not a good idea to ask for installation (twice) and fall into a fatal error if you answer is NO. There should be alternatives to patching the working copy (e.g. nightly dump etc.)
Thank you very much for this info! I will look at this as soon as possible!
The question is how to handle the case you descibed? This questions have to be regarded as final part of the install/setup process (checkout, clone or download), so how to handle a sequence of N(o) answers? In fact this means that the user does not want to finish the install/setup process - this is somehow a paradox... Shall I simply output a message:
"Install process could not be finished, framework NOT ready for use!"
Please contribute your ideas! externals/__init__.py is supposed to help us, not to bother us... :) By the way the concept is not new, is was inspired by how "VisTrails" does this.
Thanks and Greetings DrTrigon
Hi folks,
unfortunately the git Id numer is not a sequential release number as in svn but a hash. I find it a good idea to have a release number in the code as it was before git migration. A git statement generates it:
git rev-list HEAD | wc -l
which is the revision number. A better idea would be the gerrit number which gives us the last and actual change for that code. It is posible to merge such a number to the __version__ header of the script instead or in addition to the hash?
Greetings
xqt
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 01.08.2013 13:33, info@gno.de wrote:
Hi folks,
unfortunately the git Id numer is not a sequential release number as in svn but a hash. I find it a good idea to have a release number in the code as it was before git migration. A git statement generates it:
git rev-list HEAD | wc -l
which is the revision number. A better idea would be the gerrit number which gives us the last and actual change for that code. It is posible to merge such a number to the __version__ header of the script instead or in addition to the hash?
I looked this up already. The answer is given in [1] (and more in [3] and [2]), in short:
1.) You probably don't want to do this. Using git describe is a reasonable alternative. 2.) If you do need to do this, $Id$ and $Format$ are fairly easy. 3.) Anything more advanced will require using gitattributes and a custom filter. I provide an example implementation of $Date$.
That was the reason why I activated $Id$ ("fairly easy"). As I understand, what you want to do (I also would like to see ;) needs "using gitattributes and a custom filter" in [1] somebody provided an example implementation of $Date$. I strongly support this, but assmue we might need to use another keyword than $Id$, e.g. $Rev$ or $Ver$?
[1] http://stackoverflow.com/questions/39742/does-git-have-anything-like-svn-pro... [2] http://stackoverflow.com/questions/11534655/git-keyword-substitution-like-th... [3] http://git-scm.com/book/en/Customizing-Git-Git-Attributes#Keyword-Expansion
Greetings DrTrigon
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 01.08.2013 14:13, Dr. Trigon wrote:
On 01.08.2013 13:33, info@gno.de wrote:
Hi folks,
unfortunately the git Id numer is not a sequential release number as in svn but a hash. I find it a good idea to have a release number in the code as it was before git migration. A git statement generates it:
git rev-list HEAD | wc -l
which is the revision number. A better idea would be the gerrit number which gives us the last and actual change for that code. It is posible to merge such a number to the __version__ header of the script instead or in addition to the hash?
I looked this up already. The answer is given in [1] (and more in [3] and [2]), in short:
1.) You probably don't want to do this. Using git describe is a reasonable alternative. 2.) If you do need to do this, $Id$ and $Format$ are fairly easy. 3.) Anything more advanced will require using gitattributes and a custom filter. I provide an example implementation of $Date$.
That was the reason why I activated $Id$ ("fairly easy"). As I understand, what you want to do (I also would like to see ;) needs "using gitattributes and a custom filter" in [1] somebody provided an example implementation of $Date$. I strongly support this, but assmue we might need to use another keyword than $Id$, e.g. $Rev$ or $Ver$?
[1] http://stackoverflow.com/questions/39742/does-git-have-anything-like-svn-pro...
[2]
http://stackoverflow.com/questions/11534655/git-keyword-substitution-like-th...
[3]
http://git-scm.com/book/en/Customizing-Git-Git-Attributes#Keyword-Expansion
Meanwhile
I had time to have a close look to this and was able to implement [1] for compat and [2] for core.
[1] https://gerrit.wikimedia.org/r/77554 [2] https://gerrit.wikimedia.org/r/77555
This allows to use $pwbId$ like $Id$ before and even more to implement further such keywords if needed. The drawback is that during a checkout all files are touched and for each 2 calls to git shell command are invoked (which takes time). However if you like to enable this feature in your local repo please confer "4. optionally/opt-in further settings:" in [3] and run:
For compat: $ git config filter.rcs-keywords.smudge maintenance/rcs-keywords $ git config filter.rcs-keywords.clean 'maintenance/rcs-keywords --clean'
and for core the same, except you have to omit "maintenance/". See [3].
[3] http://www.mediawiki.org/wiki/Manual:Pywikipediabot/Gerrit#Example_.28step-b...
Greetings and enjoy! DrTrigon
Hi xqt and dr. trigon,
On 1 August 2013 13:33, info@gno.de wrote:
Hi folks,
unfortunately the git Id numer is not a sequential release number as in svn but a hash. I find it a good idea to have a release number in the code as it was before git migration. A git statement generates it:
git rev-list HEAD | wc -l
which is the revision number. A better idea would be the gerrit number which gives us the last and actual change for that code. It is posible to merge such a number to the __version__ header of the script instead or in addition to the hash?
Although this is possible, I'm not sure what we gain by doing this - is the problem the length of the hash? Or do you want a number that increases? If so - why?
I see it can be useful to quickly determine how old someone's version is, but we can add logic to version.py to do that (e.g. including date). In my experience, it was never very useful to see who touched a file for the last time - and it's quickly available using git log <filename> (although this might be harder on windows?)
Merlijn
----- Original Nachricht ---- Von: Merlijn van Deen valhallasw@arctus.nl An: Pywikipedia discussion list pywikipedia-l@lists.wikimedia.org Datum: 10.08.2013 16:21 Betreff: Re: [Pywikipedia-l] Release number under git/gerrit
Hi xqt and dr. trigon,
On 1 August 2013 13:33, info@gno.de wrote:
Hi folks,
unfortunately the git Id numer is not a sequential release number as in svn but a hash. I find it a good idea to have a release number in the
code
as it was before git migration. A git statement generates it:
git rev-list HEAD | wc -l
which is the revision number. A better idea would be the gerrit number which gives us the last and actual change for that code. It is posible to merge such a number to the __version__ header of the script instead or in addition to the hash?
Although this is possible, I'm not sure what we gain by doing this - is the problem the length of the hash? Or do you want a number that increases? If so - why?
I see it can be useful to quickly determine how old someone's version is, but we can add logic to version.py to do that (e.g. including date). In my experience, it was never very useful to see who touched a file for the last time - and it's quickly available using git log <filename> (although this might be harder on windows?)
Merlijn
The has is unwieldy and in past there where a lot of cases like "... it worked with release 3815 but it failed after 4711". Unfortunately the repository log only shows the hash and not any chronological number. I believe I still perserve in the old svn workflow and I probably have to rethink these things.
xqt
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Although this is possible, I'm not sure what we gain by doing this
- is the problem the length of the hash? Or do you want a number
that increases? If so - why?
The point is revision numbers are slightly more useful than hashes. The reason is you can compare them on < (less than) and > (bigger than) whereas hashes you can just compare for = (equality). So with hashes you can just say "same" revision or not, but with revisions you can also say "newer", "older" may be even "very very old"... ;))
By the way the changes [1] and [2]
[1] https://gerrit.wikimedia.org/r/77554 [2] https://gerrit.wikimedia.org/r/77555
are "opt-in" in a double manner. First you have to use $pwbId$ and second you have to enable 2 git filters in order to work.
Greetings DrTrigon
On Sat, Aug 10, 2013 at 10:21 AM, Merlijn van Deen valhallasw@arctus.nlwrote:
Hi xqt and dr. trigon,
On 1 August 2013 13:33, info@gno.de wrote:
Hi folks,
unfortunately the git Id numer is not a sequential release number as in svn but a hash. I find it a good idea to have a release number in the code as it was before git migration. A git statement generates it:
git rev-list HEAD | wc -l
which is the revision number. A better idea would be the gerrit number which gives us the last and actual change for that code. It is posible to merge such a number to the __version__ header of the script instead or in addition to the hash?
Although this is possible, I'm not sure what we gain by doing this - is the problem the length of the hash? Or do you want a number that increases? If so - why?
I see it can be useful to quickly determine how old someone's version is, but we can add logic to version.py to do that (e.g. including date). In my experience, it was never very useful to see who touched a file for the last time - and it's quickly available using git log <filename> (although this might be harder on windows?)
I'd suggest using semantic version numbers (http://semver.org/) for git annotated tags (make sure to push them to the server too) and then use `git describe` as a reasonable approximation of the SVN revision number, but with more information provided as to the significance of changes than a simple revision counter.
Tom
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I'd suggest using semantic version numbers (http://semver.org/) for git annotated tags (make sure to push them to the server too) and then use `git describe` as a reasonable approximation of the SVN revision number, but with more information provided as to the significance of changes than a simple revision counter.
Sound good - can you explain how to make this automatic?
Thanks and Greetings DrTrigon
On Sun, Aug 18, 2013 at 4:25 AM, Dr. Trigon dr.trigon@surfeu.ch wrote:
I'd suggest using semantic version numbers (http://semver.org/) for git annotated tags (make sure to push them to the server too) and then use `git describe` as a reasonable approximation of the SVN revision number, but with more information provided as to the significance of changes than a simple revision counter.
Sound good - can you explain how to make this automatic?
The version numbers need to be assigned by the developers since only they know whether it's a new patch release, minor version release, or major release.
git tag -a -m "Next major release" v3.0.0 git push origin --tags
The version number can be merged into the source by a build script or git's pseudo-keyword expansion
VERSION=`git describe` http://git-scm.com/book/ch7-2.html#Keyword-Expansion
The format that git describe uses looks like
v3.0.0-46-g71a77dc
which would be 46 commits after 3.0.0 with the given hash as the latest commit.
Tom
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
So that means we just have to
git tag -a -m "Initial release" v0.0.0 git push origin --tags
The very first revision in git and then git describe gives us the good old revision numbers back. Sounds good - how to tag the very first revision of the repo?
Then we can use git describe for $pwbId$.
Greetings
On 18.08.2013 14:52, Tom Morris wrote:
On Sun, Aug 18, 2013 at 4:25 AM, Dr. Trigon <dr.trigon@surfeu.ch mailto:dr.trigon@surfeu.ch> wrote:
I'd suggest using semantic version numbers (http://semver.org/) for git annotated tags (make sure to push them to the server too) and then use `git describe` as a reasonable approximation of the SVN revision number, but with more information provided as to the significance of changes than a simple revision counter.
Sound good - can you explain how to make this automatic?
The version numbers need to be assigned by the developers since only they know whether it's a new patch release, minor version release, or major release.
git tag -a -m "Next major release" v3.0.0 git push origin --tags
The version number can be merged into the source by a build script or git's pseudo-keyword expansion
VERSION=`git describe` http://git-scm.com/book/ch7-2.html#Keyword-Expansion
The format that git describe uses looks like
v3.0.0-46-g71a77dc
which would be 46 commits after 3.0.0 with the given hash as the latest commit.
Tom
_______________________________________________ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
So on compat:
$ git tag -a v0.0.0 -m 'Initial version' b94c23e $ git describe v0.0.0-10213-g553235e
So I have revision 10213 installed currently. If wished for I can change the implementation to one using 'git describe' instead of 'git rev-list HEAD | wc -l'... any oppinions?
May be we should call 'compat' the version v1.0.0 and 'core' the version v2.0.0.
Greetings DrTrigon
On 18.08.2013 14:52, Tom Morris wrote:
On Sun, Aug 18, 2013 at 4:25 AM, Dr. Trigon <dr.trigon@surfeu.ch mailto:dr.trigon@surfeu.ch> wrote:
I'd suggest using semantic version numbers (http://semver.org/) for git annotated tags (make sure to push them to the server too) and then use `git describe` as a reasonable approximation of the SVN revision number, but with more information provided as to the significance of changes than a simple revision counter.
Sound good - can you explain how to make this automatic?
The version numbers need to be assigned by the developers since only they know whether it's a new patch release, minor version release, or major release.
git tag -a -m "Next major release" v3.0.0 git push origin --tags
The version number can be merged into the source by a build script or git's pseudo-keyword expansion
VERSION=`git describe` http://git-scm.com/book/ch7-2.html#Keyword-Expansion
The format that git describe uses looks like
v3.0.0-46-g71a77dc
which would be 46 commits after 3.0.0 with the given hash as the latest commit.
Tom
_______________________________________________ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Le 01/08/13 13:33, info@gno.de a écrit :
Hi folks,
unfortunately the git Id numer is not a sequential release number as in svn but a hash. I find it a good idea to have a release number in the code as it was before git migration. A git statement generates it:
git rev-list HEAD | wc -l
which is the revision number. A better idea would be the gerrit number which gives us the last and actual change for that code. It is posible to merge such a number to the __version__ header of the script instead or in addition to the hash?
In git, you can mark a commit with an arbitrary text such as a sequential version numbering. You could use the usual v1.0.0 or something like 2013Q2 for quaterly releases.
Once you have created such tags, you can easily describe the ongoing version using `git describe` which lookup the most recent tags reacheable, adds the number of commit and the current sha1.
Example:
# lets create an example repository $ git init foo; cd foo Initialized empty Git repository in /tmp/foo/.git/
# generate 5 commits $ for i in 1 2 3 4 5; do git commit -m $i --allow-empty; done; [master (root-commit) f162d3d] 1 [master d2b4825] 2 [master 54ac942] 3 [master 38e4f38] 4 [master c97d909] 5 $
Here is the log:
$ git log --oneline c97d909 5 38e4f38 4 54ac942 3 d2b4825 2 f162d3d 1 $
Lets tag the two first commits (1 and 2) has respectively version v1.0 and v1.1:
$ git tag v1.0 f162d3d $ git tag v1.1 d2b4825
The log becomes:
$ git log --oneline --decorate c97d909 (HEAD, master) 5 38e4f38 4 54ac942 3 d2b4825 (tag: v1.1) 2 f162d3d (tag: v1.0) 1 $
When you checkout v1.1, git describe yields 'v1.0':
$ git checkout v1.0 ... $ git describe --tags v1.0 $
If I checkout the latest commit (text is '5'):
$ git checkout c97d909 v1.1-3-gc97d909 $
Which shows that I am ahead of the tag 'v1.1' by '3' commits and the corresponding git commit is c97d909.
cheers,
pywikipedia-l@lists.wikimedia.org