[merged thread]
Hi Bawolff and Daniel,
This explanation makes sense, as I'm running the same set of unit tests on 3 different
environments, each environment runs in parallel with one another. And I've just
realized 20191116051316 is just a timestamp with accuracy up to seconds XD
So for now I think I will just append a random suffix to the file name for this
unit-testing purpose. Thanks for your help!
Regards,
Xinyan
-----Original Message [1/2] -----
From: Wikitech-l <wikitech-l-bounces(a)lists.wikimedia.org> On Behalf Of bawolff
Sent: Tuesday, November 19, 2019 9:37 AM
To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org>
Subject: Re: [Wikitech-l] The difference between fileexists-no-change and
backend-fail-alreadyexists for action=upload
So fileexists-no-change happens when your uploading a new version of a file, MediaWiki
calculates the sha1 sum, and notices that the img_sha1 field in the db for the current
version of the file is the same as what was just calculated for the new file. Which
probably (although not necessarily now that sha1 is broken) means the file you're
uploading is the same as the one already there.
backend-fail-alreadyexists basically means that MediaWiki was about to write a file
somewhere, but there already was a file at that place, and the overwrite/overwriteSame
flags were not set, so there should not have already been a file in that place (The file
already there, may or may not be the same as the new file). Normally I would assume that
something more frontend-y would give an error before this error would be encountered.
Perhaps it might be encountered during a race condition. This is mostly speculation as
I'm not super familiar with the FileBackend code.
--
bawolff
-----Original Message [2/2] -----
From: Daniel Kinzler <dkinzler(a)wikimedia.org>
Sent: Monday, November 18, 2019 8:56 PM
To: Chen Xinyan <cxuesongc(a)outlook.com>
Subject: Re: [Wikitech-l] The difference between fileexists-no-change and
backend-fail-alreadyexists for action=upload
Hi Xinyan!
I may be wrong about this, but from my understanding backend-fail-alreadyexists is an
unexpected internal error that indicates a conflict in the storage backend. My best guess
is a problem with the granularity of the timestamp used to construct the filename - I
could imagine that, if you manage to upload two versions of a file within the same second,
the backend fails because the versions are distinguished only by timestamp (20191116051316
is 2019-11-16 05:13:16). An alternative explanation might be a race conidtion triggered
when two processes are trying to upload a file at the same time - this should be prevented
by applying the appropriate locking, but who knows.
This is all just an "informed guess", my understanding on how the file backend
works is superficial.
HTH
Daniel
On Sat, Nov 16, 2019 at 6:51 AM Chen Xinyan <cxuesongc(a)outlook.com> wrote:
Hi there,
Hope I've found the correct list for asking this question.
I was setting up a CI test case for my MediaWiki Client Library to run
a bot that will upload a file to
https://en.wikipedia.beta.wmflabs.org
in chunked stash mode. For most of the time, when I perform the final
upload with filekey parameter, I will receive a fileexists-no-change
error from the MW API server. This is expected, as there is already a
same file with the same title on the site. However, today I received
backend-fail-alreadyexists. The error message looks like
backend-fail-alreadyexists: The file
"mwstore://local-swift-eqiad/local-public/archive/9/95/20191116051316!Test_image.jpg"
already exists.
Does this error usually occurs when user tries to upload the same file?
What's the difference between fileexists-no-change and
backend-fail-alreadyexists?
Thanks,
Xinyan
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l