Correction: the train will resume on Monday since
there are no deployments
on Fridays.
On Thu, Jun 22, 2017 at 6:49 PM, Mukunda Modell <mmodell(a)wikimedia.org>
wrote:
Starting today, immediately after SWAT,
thcipriani noticed a new error in
fatalmonitor which appeared to coincide with running scap. The curious
thing is that scap only affected localization, no code was changed.
At that point I took over for thcipriani and filed an UBN bug:
https://phabricator.wikimedia.org/T168681
The error message:
Argument 1 passed to DataValues\UnboundedQuantityValue::newFromArray()
must be an instance of array, string given in extensions/Wikidata/vendor/dat
a-values/serialization/src/Deserializers/DataValueDeserializer.php on
line 141
After monitoring the errors in kibana for a while I noticed a pattern:
the error rate is changing in a very consistent way. The graph of errors
makes an almost perfect sine wave, peaking at about 150 per minute and then
bottoming out at zero. This pattern has repeated all afternoon. It seems
like such a consistent pattern could only be caused by either a) a queued
job or b) an api client. Since the bug is reproducible via the API, I
suspect that it's b.
As of now it I believe this error to be unrelated to any recent code
deployments. The timing might have just been coincidence. If that is the
case then it should not block the train any further, however, Greg made the
call to postpone further deployments until tomorrow.
If you have any input on this error, please comment on the phabricator
task. You can also ping me on IRC, I will be around for much of the evening.
Thanks, that is all I've got for now.
- Mukunda
_______________________________________________
Ops mailing list
Ops(a)lists.wikimedia.org