EEVblog Electronics Community Forum

Products => Computers => Programming => Topic started by: RoGeorge on November 08, 2019, 12:13:04 pm

Title: [SOLVED] Python application distributed as a single standalone executable file
Post by: RoGeorge on November 08, 2019, 12:13:04 pm
Is it possible to pack a python script/application + Python interpreter in a single file that will just run on another machine that does not have Python already installed?

To be more precise:
- I have a python script I made, called 'instrument.py', running fine on my dev machine
- I want to run this script on another clean machine, that does not have Python installed, and doesn't have internet
- preferably would be a "instrument.exe" file that is self contained, and just run on any other clean machine
- could be a different file for each OS, i.e. an "instrument.exe" for windows and another file "instrument" for Linux would be just fine

Is it possible to distribute a Python application as a single executable file?  How?

[SOLVED]
Code: [Select]
# install pip
sudo apt-get update
sudo apt-get install python-pip

# install PyInstaller
pip install pyinstaller

# turn a python script into a standalone executable file
pyinstaller my_python_code.py --onefile --clean

# to run the standalone executable file
./dist/my_python_code
Title: Re: Python application distributed as a single standalone executable file
Post by: mariush on November 08, 2019, 03:15:56 pm
You can make an installer for your application, a single executable.
It will ask user for the destination folder and then extract the files there.

See InnoSetup for example: http://www.jrsoftware.org/isinfo.php (http://www.jrsoftware.org/isinfo.php)
or the more basic (interface wise) NSIS : https://nsis.sourceforge.io/Main_Page (https://nsis.sourceforge.io/Main_Page)

As for making the script an exe, afaik Python is an interpreted language.

You can bundle your script with the minimal stuff required for the script to run (some kind of python runtime) and you can make a sort of very small executable (50-100 KB) which simply starts that runtime with your script as parameter.

I don't know how to do it, but a good example of an application made using Python is calibre: https://calibre-ebook.com/ (https://calibre-ebook.com/)

The author has a detailed build instructions here - https://github.com/kovidgoyal/build-calibre (https://github.com/kovidgoyal/build-calibre) -  but it's a really complicated application with lots of dependencies, so I'm not sure how the actual executables are created by the build process.

Googling results in a recommendation for PyInstaller, maybe this would be enough for your needs:

official page: https://www.pyinstaller.org/ (https://www.pyinstaller.org/)

some articles:

https://ourcodeworld.com/articles/read/273/how-to-create-an-executable-exe-from-a-python-script-in-windows-using-pyinstaller (https://ourcodeworld.com/articles/read/273/how-to-create-an-executable-exe-from-a-python-script-in-windows-using-pyinstaller)
https://datatofish.com/executable-pyinstaller/ (https://datatofish.com/executable-pyinstaller/)
 
Title: Re: Python application distributed as a single standalone executable file
Post by: RoGeorge on November 08, 2019, 03:54:13 pm
I've already tried PyInstaller, and it worked only for the native Python installation.  If I'm using Python virtual environments, or conda, PyInstaller fails.

Will look at the other methods you mentioned, thank you.

In the meantime I learned how to build Python from sources.  This way I can end up with a Python contained in a single folder, and can run it from that folder.  Additional modules can be installed with pip locally, in the built Python.  Thus, I can end up with everything in a single folder, zip that folder and unzip it later to another machine, and it should run without any installation.  If launched by a local script that defines PYTHONPATH environment variable, it should run in that folder only, without going to other eventual Python installations that might coexist on the same machine.

Not sure if this method is the proper way to do it.
Title: Re: Python application distributed as a single standalone executable file
Post by: mariush on November 08, 2019, 04:09:49 pm
I was considering learning Python but I've decided to learn Go (Golang) first : https://golang.org/ (https://golang.org/)

 
It's quite easy to learn and has built in concurrency stuff that makes it easier to make applications that do lots of things at the same time.

On top of that, it compiles everything statically as native windows or linux executable, without any runtimes required (makes the actual exe a bit big because of this though, around 500 KB after compressing it with upx)
 
The biggest downside is that it doesn't have any native GUI libraries or functions, by default you can make only command prompt applications.
There are packages which makes it possible to use qtWidgets, gtk , and other things and create windows and menus and stuff.

It's also super trivial to use the built in web server to simply create a local web server serving a single website using javascript and css and html and serve the files from within the executable and javascript in browser can exchange packets of data with the server through websockets or json or other means.

There's also packages that work like electronJS (basically a miniature web browser that creates your application window)

https://www.youtube.com/watch?v=YS4e4q9oBaU (https://www.youtube.com/watch?v=YS4e4q9oBaU)
Title: Re: Python application distributed as a single standalone executable file
Post by: SiliconWizard on November 08, 2019, 04:14:27 pm
A minimal Python runtime will still be gigantic, right? Do you have any figure to get an idea?
Title: Re: Python application distributed as a single standalone executable file
Post by: GeorgeOfTheJungle on November 08, 2019, 04:58:11 pm
Quote from: mariush on November 08, 2019, 04:09:49 pm
There's also packages that work like electronJS (basically a miniature web browser that creates your application window)
But that's javascript (nodejs). Atom works like that too ( https://atom.io/ )
Title: Re: Python application distributed as a single standalone executable file
Post by: RoGeorge on November 08, 2019, 05:37:26 pm
Quote from: SiliconWizard on November 08, 2019, 04:14:27 pm
A minimal Python runtime will still be gigantic, right? Do you have any figure to get an idea?

About 50 MB zipped and about 200 MB after decompression.  That is for a Python 3.5.9 build, without cutting out any standard (but unused) libraries.

By carefully sorting out all the unused libraries, the size could probably go 10 times smaller or so, but that would be the job of a tool, like PyInstaller.

Maybe PyInstaller can run correctly on a built from sources python (but without virtual environments), should try that.  I just found out that there is a way to tell PyInstaller where to look and where not to look when using venv, but that would be yet another thing to maintain manually, so a showstopper for using PyInstaller.
Title: Re: Python application distributed as a single standalone executable file
Post by: dferyance on November 08, 2019, 05:55:02 pm
Quote from: mariush on November 08, 2019, 04:09:49 pm
...On top of that, it compiles everything statically as native windows or linux executable, without any runtimes required...

I agree this is a good feature of Go, but also really odd that it is touted as a reason to use it. Traditionally, programming languages always generated a single executable file. The odd one out is Visual Basic with vbrun but that was corrected when VB got a full compiler. But C, C++, D, Delphi, Asm, QuickBasic, and you name it, all generated a single executable. Most compilers had an option to dynamically link to a runtime library instead of static linking but it was an option, not a requirement.

It is one thing that baffles me regarding the adoption of Docker. Docker helps you if you have tons of different file dependencies in your build output. But you shouldn't have this in the first place! A mess is a mess, if you hide it in a container or not. Support for single executable output should be viewed as the expected of all programming languages, not something unique to Go.

So yeah, I don't have a good impression of Python.
Title: Re: Python application distributed as a single standalone executable file
Post by: jc101 on November 08, 2019, 06:06:37 pm
I've recently done just this, turning a python script for mangling excel spreadsheets into .rtf format files.

I used this https://pypi.org/project/auto-py-to-exe/ and it turned the small python script, a few hundred lines, into a single 5.3Mb exe file.  The end user has no python installed on their machine at all.  Embedding all the dependencies and additional python imports into the file too.

Worked great for me.
Title: Re: Python application distributed as a single standalone executable file
Post by: mariush on November 08, 2019, 06:19:56 pm
Quote from: dferyance on November 08, 2019, 05:55:02 pm
Quote from: mariush on November 08, 2019, 04:09:49 pm
...On top of that, it compiles everything statically as native windows or linux executable, without any runtimes required...

I agree this is a good feature of Go, but also really odd that it is touted as a reason to use it.

Traditionally, programming languages always generated a single executable file. The odd one out is Visual Basic with vbrun but that was corrected when VB got a full compiler. But C, C++, D, Delphi, Asm, QuickBasic, and you name it, all generated a single executable. Most compilers had an option to dynamically link to a runtime library instead of static linking but it was an option, not a requirement.


At least Visual Basic 6 had a single 1.4 MB DLL as minimum requirement.

Most c++ executables still depend on some runtime DLL files that are within Windows (msvcrt.dll for example) or you have to download and install various runtime packages  .. see visual c++ redistributable on google (https://www.google.com/search?q=visual+c%2B%2B+redistributable).
.Net is just as bad, with reliance on various .net versions to work.
As far as I know, Go has everything built into the executable. I could simply send the exe file to a person that just installed Windows and it will simply work.
Title: Re: Python application distributed as a single standalone executable file
Post by: bd139 on November 08, 2019, 06:25:41 pm
Last thing I did this I used [1] Py2exe to build a binary and shipped it with WiX [2]. Wix is a piece of shit. Find something better!


[1] http://www.py2exe.org/ (http://www.py2exe.org/)
[2] https://wixtoolset.org/ (https://wixtoolset.org/)
Title: Re: Python application distributed as a single standalone executable file
Post by: SiliconWizard on November 08, 2019, 06:55:50 pm
Quote from: RoGeorge on November 08, 2019, 05:37:26 pm
Quote from: SiliconWizard on November 08, 2019, 04:14:27 pm
A minimal Python runtime will still be gigantic, right? Do you have any figure to get an idea?

About 50 MB zipped and about 200 MB after decompression.  That is for a Python 3.5.9 build, without cutting out any standard (but unused) libraries.

Ouch. But that's without any third-party library you might be using...
So basically, distributing a Python app as stand-alone would take 200MB.  :o
(Incidentally, the JRE is about the same size on disk.)

Title: Re: Python application distributed as a single standalone executable file
Post by: bd139 on November 08, 2019, 07:22:11 pm
That’s pretty tiny. I know a company that has a binary payload of 3 gig and the software does fuck all  :palm:
Title: Re: Python application distributed as a single standalone executable file
Post by: SiliconWizard on November 08, 2019, 08:07:47 pm
Quote from: bd139 on November 08, 2019, 07:22:11 pm
That’s pretty tiny. I know a company that has a binary payload of 3 gig and the software does fuck all  :palm:

 :-DD

This is all very depressing actually...
Title: Re: Python application distributed as a single standalone executable file
Post by: bd139 on November 08, 2019, 08:44:44 pm
Oh yes indeed it’s horrible. I’m a massive proponent of “less is more”. Unfortunately I tend to work in environments where less is a word buried under the company car park with the previous consultants who jumped out of a window  :-DD
Title: Re: Python application distributed as a single standalone executable file
Post by: RoGeorge on November 09, 2019, 12:17:42 am
On a clean virtual machine with Ubuntu 18.04 LTS, PyInstaller works just fine.

Code: [Select]
pyinstaller mtx3283.py --onefile --clean
produced a standalone executable, a single file of only 4.2 MB.   :-+
Title: Re: [SOLVED] Python application distributed as a single standalone executable file
Post by: RoGeorge on November 09, 2019, 11:29:18 am
One more thing to add about PyInstaller:

At runtime, the one file executable creates a temporary folder in /tmp then delets it at exit.  This is OK for a normal application, but if the program is a command line tool that is called hundreds of times per second, creating a temp folder, unpacking, then deleting it might become a problem.  My test program is very small, but a big application would start-up slower than a native executable.

This can be mitigated if the "--onefile" argument is omitted when created the executable distribution.  Without "--onefile" switch, a distributable folder with many files in it (folder that you can later compress and distribute as a single zip file) is created instead of a single file.  In this former situation, no extra temp folders will be created, so it will startup much faster than the all in one file version.