Hi,
The new JSON dump format uses JSON maps for aliases (and many other things). A JSON map is a thing of the form { ... }. However, if an item has no aliases, the JSON dump sets its value to the empty *list* []. This is bad since it makes object-model based JSON parsers, such as Jackson, trip over this (since the parser must know what type the parsed output should have, and [] cannot be converted into a map).
It is possible that the problem occurs with other fields as well (labels, descriptions, etc.).
There are two possible fixes:
* Correctly export empty maps as {}. * Do not export empty maps at all (leave away the key).
The second is what happens in the API. It would be nice if the JSON exported by the API would be the same as the JSON exported in the dumps.
If this is news for you, I can also create a bug report.
Cheers,
Markus
Am 01.09.2014 16:37, schrieb Markus Krötzsch:
There are two possible fixes:
- Correctly export empty maps as {}.
This would be the right way to do this, but the problem is that in php, an empty map is indistinguishable from an empty list. So how do we tell the json encoder when to use what?
- Do not export empty maps at all (leave away the key).
That forces clients to always check the key first. Very annyoing. We are actually trying to fix code that would leave keys unset. I'm surprised it would still happen in the API, especially since the dump generator and the API are using the same serializer implementation. Maybe the API does some special magic when encoding to json?
So, I don't have a good solution right now...
-- daniel
Hey,
We will hopefully soon switch to our new implementation of the serialization code. I had a quick look and it seems to have the same issue though. I filed a ticket for that here: https://github.com/wmde/WikibaseDataModelSerialization/issues/77
This would be the right way to do this, but the problem is that in php, an
empty map is indistinguishable from an empty list. So how do we tell the json encoder when to use what?
Cast the array to object.
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate Evil software architect at Wikimedia Germany ~=[,,_,,]:3
On 01.09.2014 17:06, Jeroen De Dauw wrote:
Hey,
We will hopefully soon switch to our new implementation of the serialization code. I had a quick look and it seems to have the same issue though. I filed a ticket for that here: https://github.com/wmde/WikibaseDataModelSerialization/issues/77
Thanks.
This would be the right way to do this, but the problem is that in php, an empty map is indistinguishable from an empty list. So how do we tell the json encoder when to use what?Cast the array to object.
Ok, we are trying to work around this somehow for now, but it's quite tricky to do this in a typed programming language where maps and arrays are completely different things. There are some great libraries that can parse fixed format JSON into fixed class objects, which gives you very clean code, but it only works well if the types of values are known ot some extent. If this is not given, you start coding manual parse methods again to add special cases. Since maps occur everywhere in the JSON, and all of them could sometimes be empty, this affects a lot of code.
Markus
Am 01.09.2014 17:06, schrieb Jeroen De Dauw:
Cast the array to object.
Hm, true. We really don't want objects in our "serialized" representations, but i guess empty objects to represent empty maps would be acceptable. But it's a bit scary, since PHP code looking at the structure may encounter an object where it expects an array. "This part of the structure is an associative array, unless it's empty, then it's an object" - ew...
Anyway, do you have a good idea when and where to apply that cast?
Also, can the API formatter code handle this?
-- daniel
On 01.09.2014 17:06, Jeroen De Dauw wrote:
Hey,
We will hopefully soon switch to our new implementation of the serialization code. I had a quick look and it seems to have the same issue though. I filed a ticket for that here: https://github.com/wmde/WikibaseDataModelSerialization/issues/77
This would be the right way to do this, but the problem is that in php, an
empty map is indistinguishable from an empty list. So how do we tell the json encoder when to use what?
This "language" never ceases to "amaze" me. <Obligatory PHP rant>
Cast the array to object.
Nope. Java uses type ereasure. When reading as a simple Object, all type information get lost and even casting them back only results into what the Json deserializer sees: some map with bunches of strings in it. All intermediate type information (which we have because OOP) get lost in the process.
So, I will implement a workaround, but this should get fixed, since it negates all performance gains we have by employing an advanced Json parser.
Geronimo, Fredo
On 01.09.2014 18:27, Fredo Erxleben wrote:
On 01.09.2014 17:06, Jeroen De Dauw wrote:
Hey,
We will hopefully soon switch to our new implementation of the serialization code. I had a quick look and it seems to have the same issue though. I filed a ticket for that here: https://github.com/wmde/WikibaseDataModelSerialization/issues/77
This would be the right way to do this, but the problem is that in php, an
empty map is indistinguishable from an empty list. So how do we tell the json encoder when to use what?
This "language" never ceases to "amaze" me.
<Obligatory PHP rant> >> > Cast the array to object. Nope. Java uses type ereasure. When reading as a simple Object, all type information get lost and even casting them back only results into what the Json deserializer sees: some map with bunches of strings in it. All intermediate type information (which we have because OOP) get lost in the process.
That answer was referring to the PHP side of things ;-) But still interesting.
So, I will implement a workaround, but this should get fixed, since it negates all performance gains we have by employing an advanced Json parser.
Yes, I have been worried about this too.
Markus
Geronimo, Fredo
Wikidata-tech mailing list Wikidata-tech@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-tech
wikidata-tech@lists.wikimedia.org