Node.JS – advice sought from Java Expert

[Neil WyattModerator @neilwyatt]

Bit of a shot in the dark this!

I have found a package that will convert SHP files to STL files, but it works by running some javascript in Node.JS

I have managed to install Node.js and load the 'package', but I am not sure I am running the code properly – specifically, the example code has no file path in it.

Is there a Java expert in the house?

Neil

[Neil WyattModerator @neilwyatt]

[Andrew EvansParticipant @andrewevans67134]

Is it this Neil – http://dougmccune.com/blog/2014/12/30/using-shp2stl-to-convert-maps-to-3d-models/ ? If it is then i would imagine the file path would be the same directory you have the package installed.

This is javascript – confusingly not related to java

I know node.js so if you need advice let me know.

Andy

[Neil WyattModerator @neilwyatt]

Thanks Andy

Yes, it is that code. I have lots of shp files, some of which I would dearly like to print as 3D models (Including a heat island map)

I must admit, I'm not sure I installed the package properly.

I just put:

npm install shp2stl

Into a run box. It did lots of things, I have no idea if it found the package I downloaded or if it downloaded it again, the node.js box is so small I can't read the text, even if it wasn't flying past at a rate of knots. It also closes when finished so I can't see what's happened!

I just put the code and the shp file (and the associated files, but not the stl or package) all in the same directory, associate.js with node.js, clicked the program and let it run. It did lots, but no stl file appeared. Again, I couldn't follow what was happening – is there a way to 'step' node.js or to get it to stay open at the end of execution?

Edited By Neil Wyatt on 30/04/2017 15:31:05

[SillyOldDufferModerator @sillyoldduffer]

Which operating system are you using Neil?

Dave

[Neil WyattModerator @neilwyatt]

Windows, I have installed the Windows 64 version of Node.js

Neil

[Rik ShawParticipant @rikshaw]

I node.js as well – why not ask him? He seems to know pretty much about everything!

Rik

[Neil WyattModerator @neilwyatt]

I'll discretely ignore Rik…

It's occurred to me that I should install the other 'packages' used by shp2stl.

I had assumed the shp2stl package would contain all the needed bits… but maybe not.

I have figured out that 'packages' are the javascript* equivalent of of DLLs or Subroutines held ona remote server called the 'npm registry'.

Neil

*OK got that…

[Neil WyattModerator @neilwyatt]

.. without success.

Main question, how can I run node.js without the window closing at the end?

I have no idea if anything is working or throwing errors!

Neil

[SillyOldDufferModerator @sillyoldduffer]

Not going well here. I downloaded shp2stl and the example earthquake data. Running it on Ubuntu fails almost immediately because of a JSON dependency failure, so I haven't even got as far as Neil.

What I was going to suggest was to start a terminal session and redirect the output to a file. This is easy on MAC/Linux because the shell does all that as standard. Windows cmd.exe isn't that smart so you have to download and install Microsoft PowerShell, which has all the necessary goodies.

From the command line, enter 'node shp2stl > log.txt'

The > redirect operator should copy the text output into a log file that can be paged through later with an editor like notepad.

There may be a twist. command line output often contains two separate streams of data, ordinary messages and error messages. The error stream is captured with 'node shp2stl 2> log.txt'

Don't shoot me if it doesn't work – I'm struggling with the JSON error and haven't tested the idea on Windows yet.

Dave

PS Try running cmd.exe from the run box first.  That should create a persistent terminal from which you can run node.  The buffer might be big enough to see what's what.

Edited By SillyOldDuffer on 30/04/2017 20:31:41

[SillyOldDufferModerator @sillyoldduffer]

Posted by Neil Wyatt on 30/04/2017 20:31:33:

Posted by SillyOldDuffer on 30/04/2017 20:25:25:

Don't shoot me if it doesn't work – I'm struggling with the JSON error and haven't tested the idea on Windows yet.

Dave

Does this help? (sorry about font!)

Step 1: Convert to GeoJSON

The shapefile is first read with Mike Bostock’s shapefile package, which converts the geo data into GeoJSON. This will be converted in the shapefile’s source projection, but if you want to reproject the data before converting it you can using the sourceSRS and destSRS options, which uses the ogr2ogr package to do the projection (this requires ogr2ogr already installed separately).

Umm. Could be. I've assumed the error (which is inside shp2stl) is because I have a missing JSON module. But, the documentation says JSON is built in, so I need to think again. Now where's that wet towel.

Dave

[Andrew EvansParticipant @andrewevans67134]

Node.js uses NPM (Node Package Manager) to install everything it needs for a package – so if you install the shp2stl package NPM will recursively install all packages that are required.

Try opening the command line (type cmd at the run prompt) and then going to to the directory where you installed node.

Looking at the example on GitHub (and you may already know this) – once you have the package installed you need to write some code to call the package with the appropriate options (like file names) – there is an example on the site. Also see http://stackoverflow.com/questions/6737824/how-to-run-a-hello-js-file-in-node-js-on-windows for a general example of how to run a javascript function in node.js

Hope this helps.

Andy

[Andrew EvansParticipant @andrewevans67134]

Node.js uses NPM (Node Package Manager) to install everything it needs for a package – so if you install the shp2stl package NPM will recursively install all packages that are required.

Try opening the command line (type cmd at the run prompt) and then going to to the directory where you installed node.

Looking at the example on GitHub (and you may already know this) – once you have the package installed you need to write some code to call the package with the appropriate options (like file names) – there is an example on the site. Also see http://stackoverflow.com/questions/6737824/how-to-run-a-hello-js-file-in-node-js-on-windows for a general example of how to run a javascript function in node.js

Hope this helps.

Andy

[Neil WyattModerator @neilwyatt]

Are you running his example code? he says you can't just run shp2stl, it has to sit inside a small js program.

I'm installing M$ VisualStudio Community, it supports NodeJS and I'm used to using the Atmel version of Visual Studio . it should let me do live debugging, trace etc. etc.

I think the problem is Node.js is really meant for webserver applications but he's using it as a standalong language interpreter.

The documentation for node.js on ethits website is awful – it doesn't tell you how it works at all (or even what it is in language I understand)!

Should I really be looking at javascript help?

[Andrew EvansParticipant @andrewevans67134]

Yes, node.js was originally for web applications – but it now has a huge open source ecosystem so that it is commonly used for many other purposes. For example, the shp2stl packages uses lots of preexisting coordinate and shape packages written by others and freely shared – the down side of this, as you have found, is that its a steep learning curve.

Using Visual Studio may help – but its probably over-kill for you.

I would try to get the example transformation working first.

If you are still struggling tomorrow maybe post a file over to me and i will have a go and transforming it.

[SillyOldDufferModerator @sillyoldduffer]

OK, I have progress on Ubuntu. The complicated JSON error is caused simply because the input file is not found. The code doesn't check that the file exists before attempting to convert it, so it crashes! This fits with Neil's first post 'specifically, the example code has no file path in it.'

The path is needed. From where I installed the package and the example files, the path is:

var file = '3dmaps-master/sfPopulation/SanFranciscoPopulation.shp';

Also, on Ubuntu at least:

• npm has to be run with administrator privilges
• I had to npm install all the packages mentioned in the blog plus fs. They were fs shapefile topojson three deepcopy point-in-polygon & ogr2ogr

The output is promising until the example script (I called it eq.js) throws another exception and fails:

node eq.js
bounds: 5979385.645656869 2085840.5000395626 6024741.001443654 2131293.9999859035 (cartesian)
pre-quantization: 0.00045355356240339045 0.0004545350040087592
topology: 1698 arcs, 37009 points
topojson done
simplification: effective minimum area 0.00000299
simplification: retained 29608 / 37009 points (80%)
making top planes
making side planes
making bottom planes

/home/dave/js/node_modules/shp2stl/node_modules/three/three.js:32605
for ( i = 0, il = triangles.length; i < il; i ++ ) {
^
TypeError: Cannot read property 'length' of null
at Object.THREE.Shape.Utils.triangulateShape (/home/dave/js/node_modules/shp2stl/node_modules/three/three.js:32605:30)

Might be a data problem; I'll look at it tomorrow. There is a debugger but I haven't sussed out how to use it.

Dave

[Neil WyattModerator @neilwyatt]

Posted by Andrew Evans on 30/04/2017 21:23:49:

Node.js uses NPM (Node Package Manager) to install everything it needs for a package – so if you install the shp2stl package NPM will recursively install all packages that are required.

Try opening the command line (type cmd at the run prompt) and then going to to the directory where you installed node.

Looking at the example on GitHub (and you may already know this) – once you have the package installed you need to write some code to call the package with the appropriate options (like file names) – there is an example on the site. Also see http://stackoverflow.com/questions/6737824/how-to-run-a-hello-js-file-in-node-js-on-windows for a general example of how to run a javascript function in node.js

Hope this helps.

Andy

Thanks Andy,

It's very strange, despite telling it I had put the file in the root C:, it insists on looking for it in C;/users/Neil/

Anyways, i put it where it thought it should be, and like Dave, I'm getting a barf on JSON 'cannot read property st features undefined' or something like that, then many other errors.

Ho Hum.

neil

[SillyOldDufferModerator @sillyoldduffer]

Got distracted but looking at it again now.

Might be helpful to know that Object languages (like Javascript) often produce complicated error messages from simple cases. I'm looking at:

/home/dave/js/node_modules/shp2stl/node_modules/three/three.js:32605
for ( i = 0, il = triangles.length; i < il; i ++ ) {
^
TypeError: Cannot read property 'length' of null
at Object.THREE.Shape.Utils.triangulateShape (/home/dave/js/node_modules/shp2stl/node_modules/three/three.js:32605:30)
at THREE.ShapeGeometry.addShape (/home/dave/js/node_modules/shp2stl/node_modules/three/three.js:35513:32)
at THREE.ShapeGeometry.addShapeList (/home/dave/js/node_modules/shp2stl/node_modules/three/three.js:35458:8)
at THREE.ShapeGeometry (/home/dave/js/node_modules/shp2stl/node_modules/three/three.js:35442:7)
at THREE.Shape.makeGeometry (/home/dave/js/node_modules/shp2stl/node_modules/three/three.js:32162:17)
at /home/dave/js/node_modules/shp2stl/src/topojson2threejs.js:396:24
at Array.forEach (native)
at topojson2threeJSBottomPlanes (/home/dave/js/node_modules/shp2stl/src/topojson2threejs.js:392:20)
at topojson2threejs (/home/dave/js/node_modules/shp2stl/src/topojson2threejs.js:76:16)
at topojson2stl (/home/dave/js/node_modules/shp2stl/src/shp2stl.js:77:21)

The error backtracks though the all the function calls that preceded the exception, a presentation I think is upside down. Anyway, the actual cause may be in any of the functions listed, so I'm looking at the source code to work out what the culprit is. I suspect that the input data is missing or incorrectly terminated. The debugger would make the analysis easier if I knew how to work it: can Andrew advise?

Dave

[SillyOldDufferModerator @sillyoldduffer]

Another disappointment after thinking I was getting somewhere. I've found out how to run a debugger. There are several alternatives to choose between. I went with:

• npm -g install node-inspector
• download and install Chromium (or Chrome on Windows)
• node-debug -b script.js

Running the last command opens a window in the Chrome Browser containing a Javascript debugging tool. The browser talks to the Javascript using a web interface. You can look at the source, set breakpoints, watch variables and step though the code etc. Unfortunately shp2stl does a lot of number crunching and this takes an agonisingly long time in debug mode. Too long: I just interrupted the program after an hour at 100% cpu to let my laptop cool off.

Back to the drawing board…

Dave

[SillyOldDufferModerator @sillyoldduffer]

I think there's some error in, or incompatibility with, the input data file. The examples are few years old and I'm wondering if they don't work with recent Javascript nodes.

I modified the same script to convert the napa-earthquake shp file and, despite some warnings, got an stl output. I'm pretty sure it's corrupt though. It's half the size of the stl example from github. Also, I can open the github examples OK with FreeCAD but the version generated by stc2stl crashes it. Ho hum.

Just in case I'm hitting a Linux issue, I shall repeat on Windows 10.

Dave

[Neil WyattModerator @neilwyatt]

Oh my goodness…. I really feel bad for setting you off on this wild goose chase Dave!

Thanks for all the effort, i hope you are now doing this for your personal satisfaction not me!

Neil

[Neil WyattModerator @neilwyatt]

Oh my goodness…. I really feel bad for setting you off on this wild goose chase Dave!

Thanks for all the effort, i hope you are now doing this for your personal satisfaction not me!

Neil

[SillyOldDufferModerator @sillyoldduffer]

Yes indeed, it's nerdy pleasure. When I retired I bought a book on Javascript thinking I would have loads of spare time to learn it. Silly me.

Dave

[SillyOldDufferModerator @sillyoldduffer]

Bad news. I get exactly the same errors running the examples on Windows 10 as I did on Ubuntu. (Not surprising because Javascript is a cross-platform tool-set .)

StackExchange has posts discussing similar Javascript error messages and replies suggest the most likely cause is a mismatch between the data and the program. My best guess is that current versions of node,js have become incompatible with the examples. Age gets us all in the end.

Unless I get a sudden flash of inspiration, I've given up on this one.

Cheers,

Dave

[Neil WyattModerator @neilwyatt]

Thanks for all the time you put into this.

I have emailed the chap who wrote it, if he gets back to me with a solution, I'll let you know.

Neil

[Author]

Posts