Loading Python3 file on boot

[mmoreau]

I am relatively new to Linux. I have a Raspberry Pi 0 W. I have it boot to command line and auto login.

I am trying to have it automatically run my python code (button callbacks that call a second file to write to an e-ink display).

Methods I have tried:
rc.local
init.d
etc/profile

Code: [Select]

sudo python3 /home/pi/e-Paper-master/'RaspberryPi&JetsonNano'/python/examples/TYButton.py &
I tried crontab as well

Code: [Select]

@reboot sudo python3 /home/pi/e-Paper-master/'RaspberryPi&JetsonNano'/python/examples/TYButton.py &
On all of these except on the profile one, nothing seemed to happen. I cannot tell if it is running and it doesn't do anything. When ran under profile, it couldn't get access to the dependent libraries. I am calling the second file by

Code: [Select]

os.system('python3 Tyson.py') Should I be adding sudo to it? I can run it in the terminal without it. Also, the file path included '. is that part of the problem?

Are there any suggestions?

[james_s]

If you put sudo then it will be expecting the user to type a password so I wouldn't expect that to work. Can't you just give the script you want to execute sufficient permissions so that you don't need sudo?

[mmoreau]

I don't need a password when using sudo in terminal. It is set to auto login.

This is the error I get when it is loaded through the profile method:

Code: [Select]

Traceback (most recent call last):
  File "Tyson.py", line 12, in <module>
    from waveshare_epd import epd7in5bc
ModuleNotFoundError: No module named 'waveshare_epd'

This is actually in the second file called up(it is actually located on line 10). Why would it work when ran in terminal and not here?

I did try adding all permissions to all files in the tree. (777 -R)

It also failed to write to the file it should have(happens before it needs to execute the second file).

[greenpossum]

Normally sudo expects a terminal, so running it from crontab won't work. Why can't you run python3 from root's crontab?

[notsob]

Just a dumb question, have you checked for executable file permissions
 

[mrflibble]

Are there any suggestions?

Real quick, because time *poof* 

You don't want crontab. You want to daemonize it with an /etc/init.d entry. Pick whatever init.d example script (start-stop-daemon) is closest to what you need and modify. If your python script does anything networky, you may want to put a socat wrapper around it.

And remember kids, if not socat then apt-get install socat.

That's like if not Euler then Gauss, but for networking. 

Relevant man pages:
man start-stop-daemon
man socat

Couple of links:
https://stackoverflow.com/questions/4705564/python-script-as-linux-service-daemon
https://github.com/martinrusev/python-daemon

Crap, was going to mention /etc/init.d/skeleton, but seems that it has buggered off to another closet. But on the bright side, check the following link under the heading Further reading:
https://unix.stackexchange.com/questions/480896/whither-etc-init-d-skeleton

Anyways, hope that helps to get you started. 

[mmoreau]

Sorry, it was a typo were I had sudo in the crontab. I did not.

I have been working for the past hour to get it deamonized but I'm not having much luck. I gave the start 345 and stop 0126. I'm not sure about that. I wrote it like this.

Code: [Select]

description "TYButton.py"
author  "MMoreau"

start on runlevel [345]
stop on runlevel [0126]

chdir /home/pi/tyson101/python/examples
exec /home/pi/tyson101/python/examples/TYButton.py
I moved it because they included ' in one of the folder names and I though it might be causing problems.

but running

Code: [Select]

sudo initctl reload-configurationdidn't work. I think that article might be outdated.

[mrflibble]

Okay, I take back the advice on using a daemon.
Yes, it is still is the technical solution that fitsa the bill.
But no, it apparently is not the one that fits your support contract.

So instead lets go with whatever it is you have almost working but not quite.

I don't need a password when using sudo in terminal. It is set to auto login.

This is the error I get when it is loaded through the profile method:

Code: [Select]

Traceback (most recent call last):
  File "Tyson.py", line 12, in <module>
    from waveshare_epd import epd7in5bc
ModuleNotFoundError: No module named 'waveshare_epd'

This is actually in the second file called up(it is actually located on line 10). Why would it work when ran in terminal and not here?

I did try adding all permissions to all files in the tree. (777 -R)

It also failed to write to the file it should have(happens before it needs to execute the second file).

It fails to find that module because:
1 - waveshare_epd is not a system installed python module
2 - it works from console because you probably do somthing like "cd wheverthebleepthisthingis ; python myhappypythonscript.py"
3 - but when your script is fired up from crontab it does not have the exact same runtime environment & working directory as when you start manually from console

How about a shellscript
check whichever one exists

ls /bin/bash
exists? good, use that

Does not exist?
ls /bin/sh
exists? okay, use that

Edit a file named /home/whocares/myscript.sh:

Code: [Select]

#!/bin/bash
cd <whereverthebleepmydirectorywithpythonstuffis>
python <nameofpythonscript>
chmod 755 /home/whocares/myscript.sh

Then to check if it even works, do the following in a terminal:

Code: [Select]

cd /tmp
echo "Yes /tmp, because that for sure is the wrong working directory."
echo "Incidentally, that is a similar situation your original script running from cron finds itself in"
echo "Okay, lets test"
/home/whocares/myscript.sh
echo "Did that work? Yes? yay! No? to bad."

if it worked then you can use "/home/whocares/myscript.sh" as the name of the executable that cron should start. And obviously change /home/whocares/myscript.sh filename to whatever it should be.

And "best" *cough* test while running as root. Not because it is best practice, but because remote debugging is a pain, especially when the user emitted error code is "it didn't work".
So test everything as root, run everything as root, get yelled at for running everything as root.

And then when you have everything kinda working do either of these two:
1 - break everything again by attempting to make it less non-secure.
2 - anything else.

[greenpossum]

Running from the init system differs depending on whether it's SysV init or systemd. Raspi OS uses Debian which has moved to systemd. So I still advocate using crontab and @reboot.

The thing is a crontab environment is quite bare, it won't have the usual paths. Best to spell out pathnames in full and not depend on $PATH. Any environment variable Python needs say for searching for libraries will have to be set. It won't be running in your $HOME. It also won't have an attached terminal so any module that depends on that will fail.

[mrflibble]

The thing is a crontab environment is quite bare, it won't have the usual paths. Best to spell out pathnames in full and not depend on $PATH. Any environment variable Python needs say for searching for libraries will have to be set. It won't be running in your $HOME. It also won't have an attached terminal so any module that depends on that will fail.

Indeed. That's exactly why I said that stuff I just said.

[mmoreau]

I tried crontab again and it has been working. I think the character   '    that was in the folder name was the issue. I'm not sure why the screen manufacturer included it. Probably an issue with translation. Thanks for the help.

[greenpossum]

Something to note for next time. Output from cron jobs are mailed to the user running them. If you look in root's mailbox on that machine you'll probably find output about not finding the folder that had the ' character in the name.

[Nominal Animal]

I didn't want to post in this thread, because I'm told "my way" is way too too much work.  But, because it works well, I thought it might help someone, if I include this warning here.

Basically, I'm telling you to learn how to become a linux integrator for your hardware and software stack.


I've done a few custom Linux installations (both from scratch, and based on RHEL/CentOS/Debian).  In my experience, for an embedded appliance, it is best to just rip out the entire init system, and start from scratch.  I'm also one of those who really dislikes how systemd intervenes, so I tend to use something else than systemd.  You can also start with Devuan if you want a non-systemd Linux installation.  (See here for Devuan on armel, armhf, and ppc64el.)

I typically start with a distribution known to work on the target architecture, and install an utterly minimal system in a virtual machine, so I can construct the init system from ground up, recording which packages et cetera are needed.  Usually, my target is to be able to (remount the root partition read-write), and to update the system using the base distribution package management (apt or rpm), then remount the root partition read-only.

For the actual final build, I use a real target system with a known good install and construct the new system to a different mount (MicroSD card, typically). 

Unless you have a spinny-disk HDD, an SSD, or equivalent Flash storage, you'll want to make the root mount (core OS stuff) read-only.  When running from MicroSD cards, it makes for a much more robust system.  In any case, you'll want to use conservative mount flags (noatime, nodiratime, or relatime especially) to minimize writes in general.

This is not nearly as difficult as one might imagine, if you have a computer that can run a virtual machine.  It is absolutely fine to run x86 or x86-64 for the init replacement work, even if the target system runs on ARM.  Basically, if you start a traditional init or indeed any init variant, you only need to make sure all media is mounted, and you can have init start a login/bash session.  It sounds so simple, and it is: the current systemd stuff is mostly bloat and desire for centralized control written by rather poor programmers, and that's what I don't like about it, really; it is not ideological.  One doesn't normally understand or believe this, until they actually play with this stuff.

(One of my pet peeves is having services like ModemManager running all the time.  Why?  If I want a modem connection, the connection manager could easily ask the service to start then an there, and when the last connection closes, it could stop itself.  But nooo, that's not how we do it: we want ModemManager to run all the time, and whenever you have a serial-port like device plugged in to your USB port, ModemManager will probe it like the little rapist it is, making all sorts of USB microcontroller development on Linux harder than it should be.  Ahem. Rant over.)

A better option is to create a straightforward init system with a startup script, that starts exactly the services you need.  For embedded appliances, that ain't too many of them; a lot of the processes running on a typical Linux machine nowadays are silly little user management crap not needed on an embedded device.  You'll also want at least one getty running, so that you can log in to the system via the system console; each init system has a way to ensure it/they run on the ttys you want.

While the components must be compiled/installed separately for each hardware architecture, the exact same software configuration (aside from things like ensuring which hardware drivers need to be probed/loaded, which EGL backend Embedded Qt is using, and such) will work.  This is why starting with a distro you like that works on the hardware, but running on x86/x86-64 in a virtual machine, to get a grip how to get the system running in the way you want, is perfectly okay.

Do run SSH, but run it on a nonstandard port, and use a nonstandard username for access (not root, admin, sysadmin, etc.).  This is to stop script kiddies from breaking in in case the SSH/TLS library version you use turns out to have an exploitable security hole.  Using SSH keys is a good idea, just keep your private key private (with a backup in hard storage).

If you'll have long boot times, you can add a bootsplash.  That way you get a professional looking logo/animation during bootup.  I prefer to keep the boot times so short a static image suffices.  On current systems, if you compile your own kernels, with the display drivers built in, you can throw the bootsplash very, very early in the boot sequence; within a fraction of a second of init starting.  (And on most embedded systems, that means within a second or two of powering up the system.)

You can run Embedded Qt on top of "raw" EGL (RPi, Mali, i.MX6), if your application is the only one and is fullscreen.  You don't run Xorg or window manager or session manager, as Qt connects directly to the hardware device; you do need to configure and compile Qt for that hardware, though.  I recommend running the applications using a dedicated user and group, with udev rules granting audio and display hardware access to that group.

If you want multiple applications, you can run Xorg and a window manager you like.  (These are started as root, but the applications as dedicated users.)

If you go this route, you'll want to use SBCs that upstream their hardware support, so you can use "vanilla" kernels (from kernel.org, or from the distribution you used as a basis) and are not beholden to the hardware vendor providing binary driver updates for your hardware for the kernel version you want. (For this reason alone, I like some of the Amlogic based Linux SBCs, like La Potato, La Frite, Odroid-C1+, and Odroid-C4.  Only have two of those four, though.)

Also, if building a complete Linux distro from scratch, from sources, is something you get interested in, or you find you have to recompile a particular package for your own needs, see linuxfromscratch.org.  They have lots of information on how to build a system from scratch, and how to compile/massage some packages to work better.

[mrflibble]

I didn't want to post in this thread, because I'm told "my way" is way too too much work.  But, because it works well, I thought it might help someone, if I include this warning here.

Basically, I'm telling you to learn how to become a linux integrator for your hardware and software stack.

Bonus points for your detailed post & patience good sir! 

All I manage these days is the forum equivalent of drive by shooting shouting. You know, where you roll down the car window, shout a couple hopefully helpful remarks at that-guy-doing-python-stuff in the street, hope he gets it, and drive on because time *poof*.

Couple more short remarks, best to use a virtual environment for 3rd party libraries.
https://docs.python.org/3/tutorial/venv.html
https://docs.python.org/3/library/venv.html

Code: [Select]

sudo apt-get install python3-pip python3-venv git

Then create the actual virtual env as a regular user, not as root.

Code: [Select]

sudo su - pi
echo "Should be logged in as regular user now."
mkdir -p /home/pi/src/waveshare
cd /home/pi/src/waveshare
git clone https://github.com/waveshare/e-Paper
echo "The setup script for that waveshare thingy should be located here:"
ls -al '/home/pi/src/waveshare/e-Paper/RaspberryPi&JetsonNano/python/setup.py'

Alright, time to create virtual env, and install waveshare + depends. Still as regular "pi" user.

Code: [Select]

sudo su - pi
echo "Should be logged in as regular user now."
mkdir -p /home/pi/python-venv
cd /home/pi/python-venv
python3 -m venv waveshare_epd-py3
echo "Activate virtual env, and install required python modules:"
source /home/pi/python-venv/waveshare_epd-py3/bin/activate
pip install wheel
pip install pil numpy
pip install RPi.GPIO
echo "Time to install waveshare lib by using its setup.py script"
pip install -e '/home/pi/src/waveshare/e-Paper/RaspberryPi&JetsonNano/python/.'

And that should take care of the dependencies if I read the docs correctly.
edit: these docs: https://github.com/waveshare/e-Paper/blob/master/RaspberryPi%26JetsonNano/python/readme_rpi_EN.txt
Now you can use that virtual env from shell scripts.

Code: [Select]

#!/bin/bash

# Use waveshare virtual env. Paths for those python modules will be all set after this.
source /home/pi/python-venv/waveshare_epd-py3/bin/activate

# You still will have to specify where the python /script/ is. Obviously.
# Because TYButton.py is a script, not a module.
# Change full path accordingly:
python /home/pi/tyson101/python/examples/TYButton.py

If you use the above script, it will no longer depend on a random combination of python modules.
If you don't see the advantage, fair enough. Do a couple more python projects depending on XYZ and ABC, and then re-evaluate.