Hi Gilles, Thank you for having a look into my proposal and giving your valuable feedback. The proposal I sent was a brief outline of my project. I kept it short as I have been advised to do so. I have given answers to some of your queries with respect to the .x3d format below.
The X3D format provides an XML code as an source code. This XML script can be used to extract information of the file. There are functions like getScreenshot() which return a .png file of the 3D image. There are also functions to manipulate the camera views (given in the API documentation of x3d). Using these views we can set a standard view and use the getScreenshot() function to get a .png file. How getScreenshot() actually works is : " getScreenshot() Returns: URL to image Returns a Base64 encoded data URI containing png image consisting of the current rendering. The browser will interpret this as a PNG image and display it. A list of browsers which support data URI can be found here. The following example illustrates the usage: var url = ...runtime.getScreenshot(); var img = document.createElement("img"); img.src = url; ... " This is taken from the documentation of x3d.
Now, to texture. Every x3d file has a texture map. This map contains Background, ImageTexture, MovieTexture, MultiTexture, and PixelTexture. In all cases, texture maps are defined by 2D images that contain an array of colour values describing the texture. If we want detailled texture we can rebuild the texture using this map. The only problem I see here is that this process might not be feasible in terms of memory and speed. Maybe it will work out. Otherwise I believe even the screenshot will retain some texture and we can use that as well. The good thing here is that the texture map provided by x3d has a format in terms of PNG pixels which shall make compatibility easy. Some more details on texture maps : Texture maps are defined in a 2D coordinate system (s, t) that ranges from [0.0, 1.0] in both directions. The bottom edge of the image corresponds to the S-axis of the texture map, and left edge of the image corresponds to the T-axis of the texture map. The lower-left pixel of the image corresponds to s=0, t=0, and the top-right pixel of the image corresponds to s=1, t=1. Texture maps may be viewed as two dimensional colour functions that, given an (s, t) coordinate, return a colour value colour(s, t).
I have not studied COLLADA in much depth and will get back to you on it in some time. For the moment the advantages of using x3d seem to be : -> First of all x3d seems to be pretty well documented and shall be easy to go in even further depth.
-> All x3d files have a source code in xml which I have no problems with.
->Further, since we can get all contents of the file in xml it shall be easy to validate it as well.
-> Functions like getScreenShot() should make our process comparitively easy.
->Also, x3d files have meta statements. These meta statements provide information of the x3d scene as a whole. We can directly take this information from these meta statements.
Another thing that I found which might be relevant could be -
"*Blender export* : Converting Blender scenes into X3DOM webpages is pretty simple: Blender already supports direct X3D export even so there are some issues. Blender Version 2.4 seems to export some more nodes (e.g. lights), but in general it works. We will explore this more in the future, but an exported X3D file ( may afterwards be easily be integrated into an HTML webpage using X3DOM. Just finish your model in Blender and export to x3d file format. There are two ways to get your X3D data into the HTML page, and both include no coding at all: 1. Two-file solution, link the X3D-file Just use a very simple X3D scene in your HTML file, which, more or less, only includes an <inline> node. This nodes references and asynchronously downloads the X3D file. Therefore you need a real web server (e.g. Apache) running while using <inline> nodes in X3DOM. 2. One-file solution, embed the X3D data inside of the HTML page You can embed the X3D file by hand in a (X)HTML page, but this may include some hand-tweaking. Better use the aopt-converter described in Generic 3D data conversion. This can be done offline with a single command: aopt -i horse.x3d -N horse.html (horse.x3d is an x3d file being converted into an html one) You also may use the converter online. Just open horse.x3d with your favorite text editor and paste it into the source text field. Choose XML encoding (X3D) as input type and HTML5 encoded webpage as output type and press the Convert encoding button." Even though I am more sure about the former method, I am mentioning this here as there was some discussion on the bug related to blender files. If this works then using x3d we can incorporate blender files as well.
I would be highly grateful if you could provide me with further insights into this project.
Regards, Umang
On Fri, Mar 21, 2014 at 1:27 AM, Umang Sharma umange@gmail.com wrote:
The X3D format provides an XML code as an source code. This XML script can be used to extract information of the file. There are functions like getScreenshot() which return a .png file of the 3D image. There are also functions to manipulate the camera views (given in the API documentation of x3d). Using these views we can set a standard view and use the getScreenshot() function to get a .png file. How getScreenshot() actually works is : " getScreenshot() Returns: URL to image Returns a Base64 encoded data URI containing png image consisting of the current rendering. The browser will interpret this as a PNG image and display it. A list of browsers which support data URI can be found here. The following example illustrates the usage: var url = ...runtime.getScreenshot(); var img = document.createElement("img"); img.src = url; ... " This is taken from the documentation of x3d.
This seems to be a quote from the documentation for the X3DOM runtime api. X3DOM is a javascript library for browser-based display and manipulation of x3d content. It's not obvious to me by reading the documentation for X3DOM [0] that this library would be in any way useful for creating raster images from x3d files server-side.
On Thu, Mar 20, 2014 at 4:26 AM, Gilles Dubuc gilles@wikimedia.org wrote:
- Do X3D and COLLADA files always contain camera information? If they only
contain the geometry (and textures?) of the models, then we're probably going to have an issue with picking a default camera position.
The X3d spec defines a Viewpoint [1] node that can be used to provide a camera position. It seems that the conformance tests also expect that the default viewpoint for models which do not define one is (0 0 10) [2].
Also, another question that needs to be answered is whether textures are embedded in these files or if they are referenced. If they are referenced instead of being embedded, then texture support in itself would be a complex matter (how would people upload the images needed as textures?), and your proposal makes no mention of it at the moment.
ImageTexture nodes define a url field [3] specifying the image to download and apply. This does complicate the server side processing quite a bit. We almost certainly wouldn't want to download the texture images in response to the need to generate a thumbnail. Downloading images to the servers could be done at upload time as a job similar to the way that GWToolset downloads images based on a bulk specification file. There are problems with this however including the fact that only a limited number of external domains are whitelisted for downloading. Things get even more complicated due to the MovieTexture node that can be used to bind MPEG1-Video as the texture for a surface.
The inclusion of external content as textures seems to also raise a potential privacy concern for allowing users to view the x3d content directly in their browser via X3DOM or other javascript/HTML5 rendering techniques. If the textures are drawn from sources outside of commons and/or the bits servers it would be possible to effectively track the viewers of the models ala the classic single pixel advertising tracking bug. This becomes even more troubling if one imagines allowing embedding a x3d directly in an article page which could allow tracking viewers of that article. This leads me to believe that we would have to import all texture files to commons and rewrite the model to reference content from there rather than an external host.
[0]: http://x3dom.org/docs/dev/ [1]: http://www.web3d.org/files/specifications/19775-1/V3.3/Part01/components/nav... [2]: http://www.web3d.org/x3d/content/examples/Conformance/BindableNodes/Viewpoin... [3]: http://www.web3d.org/files/specifications/19775-1/V3.3/Part01/components/tex...
On Mar 22, 2014 6:50 PM, "Bryan Davis" bd808@wikimedia.org wrote:
On Fri, Mar 21, 2014 at 1:27 AM, Umang Sharma umange@gmail.com wrote:
The X3D format provides an XML code as an source code. This XML script
can
be used to extract information of the file. There are functions like getScreenshot() which return a .png file of the 3D image. There are also functions to manipulate the camera views (given in the API
documentation of
x3d). Using these views we can set a standard view and use the getScreenshot() function to get a .png file. How getScreenshot() actually works is : " getScreenshot() Returns: URL to image Returns a Base64 encoded data URI containing png image consisting of the current rendering. The browser will interpret this as a PNG image and display it. A list of browsers which support data URI can be found
here. The
following example illustrates the usage: var url = ...runtime.getScreenshot(); var img = document.createElement("img"); img.src = url; ... " This is taken from the documentation of x3d.
This seems to be a quote from the documentation for the X3DOM runtime api. X3DOM is a javascript library for browser-based display and manipulation of x3d content. It's not obvious to me by reading the documentation for X3DOM [0] that this library would be in any way useful for creating raster images from x3d files server-side.
On Thu, Mar 20, 2014 at 4:26 AM, Gilles Dubuc gilles@wikimedia.org
wrote:
- Do X3D and COLLADA files always contain camera information? If they
only
contain the geometry (and textures?) of the models, then we're probably going to have an issue with picking a default camera position.
The X3d spec defines a Viewpoint [1] node that can be used to provide a camera position. It seems that the conformance tests also expect that the default viewpoint for models which do not define one is (0 0 10) [2].
Also, another question that needs to be answered is whether textures are embedded in these files or if they are referenced. If they are
referenced
instead of being embedded, then texture support in itself would be a
complex
matter (how would people upload the images needed as textures?), and
your
proposal makes no mention of it at the moment.
ImageTexture nodes define a url field [3] specifying the image to download and apply. This does complicate the server side processing quite a bit. We almost certainly wouldn't want to download the texture images in response to the need to generate a thumbnail. Downloading images to the servers could be done at upload time as a job similar to the way that GWToolset downloads images based on a bulk specification file. There are problems with this however including the fact that only a limited number of external domains are whitelisted for downloading. Things get even more complicated due to the MovieTexture node that can be used to bind MPEG1-Video as the texture for a surface.
The inclusion of external content as textures seems to also raise a potential privacy concern for allowing users to view the x3d content directly in their browser via X3DOM or other javascript/HTML5 rendering techniques. If the textures are drawn from sources outside of commons and/or the bits servers it would be possible to effectively track the viewers of the models ala the classic single pixel advertising tracking bug. This becomes even more troubling if one imagines allowing embedding a x3d directly in an article page which could allow tracking viewers of that article. This leads me to believe that we would have to import all texture files to commons and rewrite the model to reference content from there rather than an external host.
[1]:
http://www.web3d.org/files/specifications/19775-1/V3.3/Part01/components/nav...
[2]:
http://www.web3d.org/x3d/content/examples/Conformance/BindableNodes/Viewpoin...
[3]:
http://www.web3d.org/files/specifications/19775-1/V3.3/Part01/components/tex...
-- Bryan Davis Wikimedia Foundation bd808@wikimedia.org [[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID USA irc: bd808 v:415.839.6885 x6855
Multimedia mailing list Multimedia@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/multimedia
We could maybe make it data: urls allowed only like we do with svgs.
As for movie textures - that seems like a big bag of worms. If that feature isnt commonly used, maybe we should just disallow files using it.
-- Bawolff
I think that data: uris work for SVG because SVGs with references are a lot less common than "plain" SVGs. And when they're used the references have generally created by the same author. In the case of 3d models, textures are the norm, especially textures created by someone else. Therefore, we would need the uploader to enter licensing information for each texture anyway.
I agree with Bryan that this would need to be handled on the client side at upload time. Basically upload a 3d model, each texture is displayed for entering licensing information, then everything is uploaded by the client (there might be issues with CORS there if we want to avoid the server ever downloading arbitrary URLS), and the XML image references are updated to point to textures we host. Or they're converted to data URIs. While that later option is simpler for us, it also means that a model would take longer to display at all. External references allow you to display a model untextured at first, while the textures are downloaded by the client. With data: URIs you lose that benefit.
With the upcoming major work we're about to embark on regarding UploadWizard and the upload pipeline, I think that this project, which is so unlike the existing upload flow, would be badly timed. If written based on the existing UploadWizard, it will be very unlikely to be merged back by the time we've cleaned up or rewritten UW entirely.
I also think that there might be too many big problems with major decisions for it to be suitable for GSOC. Complex issues both on the frontend and the backend. Unless we consider that anything that gets built will only be useful for improving our common knowledge of the format for a future real integration, knowing that anything that gets built by Umang won't be merged. Is it useful to do what is essentially a throwaway project, though? I'm not sure how to even scope it in that context.
Maybe the project would be to support viewing as many 3d formats as possible on the file page using JS, in a reusable fashion that could later be integrated into Media Viewer. The upload pipeline would be totally ignored, meaning that this work wouldn't get merged, but would be useful whenever we embark on making 3d model upload possible. Maybe it's too thin a project, though, as most of the code would be already written as 3rd-party 3d model viewers.
On Sunday, March 23, 2014, Brian Wolff bawolff@gmail.com wrote:
On Mar 22, 2014 6:50 PM, "Bryan Davis" bd808@wikimedia.org wrote:
On Fri, Mar 21, 2014 at 1:27 AM, Umang Sharma umange@gmail.com wrote:
The X3D format provides an XML code as an source code. This XML script
can
be used to extract information of the file. There are functions like getScreenshot() which return a .png file of the 3D image. There are
also
functions to manipulate the camera views (given in the API
documentation of
x3d). Using these views we can set a standard view and use the getScreenshot() function to get a .png file. How getScreenshot() actually works is : " getScreenshot() Returns: URL to image Returns a Base64 encoded data URI containing png image consisting of
the
current rendering. The browser will interpret this as a PNG image and display it. A list of browsers which support data URI can be found
here. The
following example illustrates the usage: var url = ...runtime.getScreenshot(); var img = document.createElement("img"); img.src = url; ... " This is taken from the documentation of x3d.
This seems to be a quote from the documentation for the X3DOM runtime api. X3DOM is a javascript library for browser-based display and manipulation of x3d content. It's not obvious to me by reading the documentation for X3DOM [0] that this library would be in any way useful for creating raster images from x3d files server-side.
On Thu, Mar 20, 2014 at 4:26 AM, Gilles Dubuc gilles@wikimedia.org
wrote:
- Do X3D and COLLADA files always contain camera information? If they
only
contain the geometry (and textures?) of the models, then we're probably going to have an issue with picking a default camera position.
The X3d spec defines a Viewpoint [1] node that can be used to provide a camera position. It seems that the conformance tests also expect that the default viewpoint for models which do not define one is (0 0 10) [2].
Also, another question that needs to be answered is whether textures
are
embedded in these files or if they are referenced. If they are
referenced
instead of being embedded, then texture support in itself would be a
complex
matter (how would people upload the images needed as textures?), and
your
proposal makes no mention of it at the moment.
ImageTexture nodes define a url field [3] specifying the image to download and apply. This does complicate the server side processing quite a bit. We almost certainly wouldn't want to download the texture images in response to the need to generate a thumbnail. Downloading images to the servers could be done at upload time as a job similar to the way that GWToolset downloads images based on a bulk specification file. There are problems with this however including the fact that only a limited number of external domains are whitelisted for downloading. Things get even more complicated due to the MovieTexture node that can be used to bind MPEG1-Video as the texture for a surface.
The inclusion of external content as textures seems to also raise a potential privacy concern for allowing users to view the x3d content directly in their browser via X3DOM or other javascript/HTML5 rendering techniques. If the textures are drawn from sources outside of commons and/or the bits servers it would be possible to effectively track the viewers of the models ala the classic single pixel advertising tracking bug. This becomes even more troubling if one imagines allowing embedding a x3d directly in an article page which could allow tracking viewers of that article. This leads me to believe that we would have to import all texture files to commons and rewrite the model to reference content from there rather than an external host.
We could maybe make it data: urls allowed only like we do with svgs.
As for movie textures - that seems like a big bag of worms. If that feature isnt commonly used, maybe we should just disallow files using it.
-- Bawolff
Sir, Can we not keep uploading of 3D models separate from your upload pipeline until we are confident on how to merge it? This way whatever will be changed would be in the existing features which you are anyways going to change. This minimizes any risk you might think up of. Further, If you provide a insight on how you plan to implement the upload pipeline, maybe we can work on this keeping that in mind and ensure to keep as much relevance as we can between the two. I am not saying the two will be in complete sync, but we can atleast work on it in a way that there are less compatibility issues later. This way the project also has more content and is not as thin as before. Maybe with little effort we might be able to get this project merged with the pipeline. Since this project being a long standing popular request, perhaps it is better to start some work on this. However, is there a possibility to build this on the basis of the new upload wizard? If not, should there not be atleast some method to upload 3D models (even though it works in a different manner than the future upload methods) until a new way gets figured out?
Regards, Umang
On Fri, Mar 21, 2014 at 7:27 AM, Umang Sharma umange@gmail.com wrote:
Hi Gilles, Thank you for having a look into my proposal and giving your valuable feedback. The proposal I sent was a brief outline of my project. I kept it short as I have been advised to do so. I have given answers to some of your queries with respect to the .x3d format below.
The X3D format provides an XML code as an source code. This XML script can be used to extract information of the file. There are functions like getScreenshot() which return a .png file of the 3D image. There are also functions to manipulate the camera views (given in the API documentation of x3d). Using these views we can set a standard view and use the getScreenshot() function to get a .png file. How getScreenshot() actually works is : " getScreenshot() Returns: URL to image Returns a Base64 encoded data URI containing png image consisting of the current rendering. The browser will interpret this as a PNG image and display it. A list of browsers which support data URI can be found here. The following example illustrates the usage: var url = ...runtime.getScreenshot(); var img = document.createElement("img"); img.src = url; ... " This is taken from the documentation of x3d.
Now, to texture. Every x3d file has a texture map. This map contains Background, ImageTexture, MovieTexture, MultiTexture, and PixelTexture. In all cases, texture maps are defined by 2D images that contain an array of colour values describing the texture. If we want detailled texture we can rebuild the texture using this map. The only problem I see here is that this process might not be feasible in terms of memory and speed. Maybe it will work out. Otherwise I believe even the screenshot will retain some texture and we can use that as well. The good thing here is that the texture map provided by x3d has a format in terms of PNG pixels which shall make compatibility easy. Some more details on texture maps : Texture maps are defined in a 2D coordinate system (s, t) that ranges from [0.0, 1.0] in both directions. The bottom edge of the image corresponds to the S-axis of the texture map, and left edge of the image corresponds to the T-axis of the texture map. The lower-left pixel of the image corresponds to s=0, t=0, and the top-right pixel of the image corresponds to s=1, t=1. Texture maps may be viewed as two dimensional colour functions that, given an (s, t) coordinate, return a colour value colour(s, t).
I have not studied COLLADA in much depth and will get back to you on it in some time. For the moment the advantages of using x3d seem to be : -> First of all x3d seems to be pretty well documented and shall be easy to go in even further depth.
-> All x3d files have a source code in xml which I have no problems with.
->Further, since we can get all contents of the file in xml it shall be easy to validate it as well.
-> Functions like getScreenShot() should make our process comparitively easy.
->Also, x3d files have meta statements. These meta statements provide information of the x3d scene as a whole. We can directly take this information from these meta statements.
Another thing that I found which might be relevant could be -
"*Blender export* : Converting Blender scenes into X3DOM webpages is pretty simple: Blender already supports direct X3D export even so there are some issues. Blender Version 2.4 seems to export some more nodes (e.g. lights), but in general it works. We will explore this more in the future, but an exported X3D file ( may afterwards be easily be integrated into an HTML webpage using X3DOM. Just finish your model in Blender and export to x3d file format. There are two ways to get your X3D data into the HTML page, and both include no coding at all:
- Two-file solution, link the X3D-file
Just use a very simple X3D scene in your HTML file, which, more or less, only includes an <inline> node. This nodes references and asynchronously downloads the X3D file. Therefore you need a real web server (e.g. Apache) running while using <inline> nodes in X3DOM. 2. One-file solution, embed the X3D data inside of the HTML page You can embed the X3D file by hand in a (X)HTML page, but this may include some hand-tweaking. Better use the aopt-converter described in Generic 3D data conversion. This can be done offline with a single command: aopt -i horse.x3d -N horse.html (horse.x3d is an x3d file being converted into an html one) You also may use the converter online. Just open horse.x3d with your favorite text editor and paste it into the source text field. Choose XML encoding (X3D) as input type and HTML5 encoded webpage as output type and press the Convert encoding button." Even though I am more sure about the former method, I am mentioning this here as there was some discussion on the bug related to blender files. If this works then using x3d we can incorporate blender files as well.
I would be highly grateful if you could provide me with further insights into this project.
Regards, Umang
multimedia@lists.wikimedia.org