[EV3DEV] Venturing into the World of EV3Dev and Python

[Jim]

12 minutes ago, MajorAlvega said:

And for visual browsing the file system I think WinSCP does that, it's like Filezilla but uses SSH instead of FTP.

Sweet! That's working like a charm.

I know this will be a journey with a lot of testing, trial and error and asking for help, but I really do like the setup so far. I love that I can access my brick and use it as a real client.

Just now, MajorAlvega said:

You will have to wait for @dwalton76 to answer... I'm just a googler wise-guy, not a real programmer :)

Hehe no problem. Your help is appreciated and you just fixed two of my issues.

I would love to be able to consume my own Web Api. That will open up lots of cool options.

10 minutes ago, Jim said:

Another question; is it possible to use the requests class in the EV3 bricks?

I do see a lot on info on how to consume a Web Api, so that won't be a problem, as long as I can get the requests library available on the brick. 

@dwalton76 any help on that would be appreciated 

[dwalton76]

Try “sudo pip3 install requests” to get that module installed 

[Jim]

6 hours ago, dwalton76 said:

Try “sudo pip3 install requests” to get that module installed 

Thanks! That almost did the trick. Had to install pip3 first.

So I ran this command to update the packages:

sudo apt-get update

And then this command to get pip3. This did take quite a while (about 15-20 minutes). I was under the impression that the entire python3 sdk/runtime was installed on my brick again.

sudo apt-get install python3-pip

And then this command to install the requests package

sudo pip3 install requests

Working like a charm! 

Now is probably the right time to start a generic Python course. I am getting really enthusiastic about Pyhton. And about Linux...for the first time in my life 

I have just been able to call my .NET Core Web Api from my EV3 brick!! 

[Jim]

13 hours ago, Jim said:

I have edited the hostname file in etc with sudoedit from ev3dev to EV3A, but it still shows ev3dev in Putty and in VSCode. Any idea why?

This problem has been solved too. Of course, a simple reboot did the trick.

[Jim]

I have created a separate topic for BrickPi General Discussion.

[dwalton76]

FYI there are some really good ev3dev python docs here:

https://sites.google.com/site/ev3devpython/

The author is looking for feedback on the site, please post here with any feedback:

https://github.com/ev3dev/ev3dev-lang-python/issues/511

[Jim]

I have found that one. I even listed it in the first post. Thanks  

I will give some feedback.

[David Lechner]

This is a matter of opinion, but `sudo pip3 install ...` is a good way to break things. Eventually, you will install a package that breaks something else that is used by the system. It is probably not so much a problem on ev3dev since there isn't much, if anything, installed by default that can be broken this way. But I think it is better to not get into this habit in the first place. (I learned this the hard way with my desktop.)

Since ev3dev is based on Debian, most popular Python packages are already available as Debian packages. So, `sudo apt update && sudo apt install python3-requests` would install the Debian version of this library. This is also really good for Python packages that include a binary component. The Debian package will contain a pre-compiled binary whereas compiling on a RPi can be tricky to install all of the correct dependencies and the EV3 has the same issues plus it is 10x slower.

If there is a Python package you want to install that does not already have a Debian package, then it is still better to install it with pip3 as a user package rather than installing it system-wide. To do this, just don't put `sudo` in front of `pip3`. Then packages will be installed to the local user directory `~/.local/lib/python3.5/site-packages/` instead of the system-wide python packages directory.

[Jim]

Hi David,

First of all, welcome to EB.

Second, thanks for the tip. I was wondering about installing packages and how it would effect the brick and the solution. Your insights are very useful. I am new to Python, but I do have .Net experience, so I know how (nuget) packages can make or break a solution, when not handled correctly.

I am in the midst of flashing two new Micro SD Cards for two of my bricks, so I will try to use the 'sudo apt' commands and see how it goes.

[Basiliscus]

Fascinating topic!

I work with Python on a daily basis but within the Windows ecosystem, so not that familiar with Linux in general.

Why would you want your EV3 to talk to a .NET API? Would you store the commands within the API and then create something that consumes the JSON in a flexible manner so you can just generate the commands dynamically?

[Jim]

48 minutes ago, Basiliscus said:

Why would you want your EV3 to talk to a .NET API?

Because I can! 

Actually, I am/was working on a .Net Core Web Api for serving high scores for my mobile games (another hobby I have). So, I wondered if I could let my bots communicate with the Api. If I can send and retrieve info to and from an Api, I can do a lot of stuff. But it is mainly a proof of concept for now. 

One thing would be to enter commands online, which the robot will polll and execute. Even better would be to be able to send commands directly from a website (or mobile phone) to the device, via Websockets or something similar. Basically, I am trying out different communication scenarios.

2 hours ago, David Lechner said:

So, `sudo apt update && sudo apt install python3-requests` would install the Debian version of this library.

Works like a charm! Thanks

49 minutes ago, Basiliscus said:

I work with Python on a daily basis but within the Windows ecosystem, so not that familiar with Linux in general.

I am a Windows guy myself and I have never touched Python, until last week. I am totally in love with the EV3, VS Code and Python combo. 

Next stop is socket communication, so two bricks communicating via a LAN.

[MajorAlvega]

On 11/16/2018 at 11:41 AM, Jim said:

Because I can! 

That's the spirit!!! :D

[Jim]

15 minutes ago, MajorAlvega said:

That's the spirit!!! :D

Where we are going we don't need roads....or reasons to access Web Api's from EV3 robots 

[Basiliscus]

51 minutes ago, Jim said:

Because I can! 

Actually, I am/was working on a .Net Core Web Api for serving high scores for my mobile games (another hobby I have). So, I wondered if I could let my bots communicate with the Api. If I can send and retrieve info to and from an Api, I can do a lot of stuff. But it is mainly a proof of concept for now. 

One thing would be to enter commands online, which the robot will polll and execute. Even better would be to be able to send commands directly from a website (or mobile phone) to the device, via Websockets or something similar. Basically, I am trying out different communication scenarios.

****

I am a Windows guy myself and I have never touched Python, until last week. I am totally in love with the EV3, VS Code and Python combo. 

Next stop is socket communication, so two bricks communicating via a LAN.

Ha! Yeah I totally get the "because I can attitude", my current side project is building a website served on my home LAN which will allow my wife to undertake searches on all of our photos - it will do face and image content recognition (I'm a data scientist) so she doesn't ask "Remember that photo of our daughter on a slide", she can search the keywords and it will find the images! I'll share the GitHub repository later when it's a bit more advanced.

Rather than using websockets you'd be better off using MQTT - set up a broker and make the web app send messages to the broker and the EV3 a subscriber. The architecture handles packet loss and general web instability a lot better and it's used a lot in IOT applications.

Whatever you do, good luck!

[Jim]

This is exactly why I created this forum. A place for people doing interesting stuff...because we can  

Thanks for the MQTT pointer. These are the things that are really helping me out. I appreciate that a lot.

Please stick around  

In the future I would like to do face recognition with a camera on my robot. Maybe you have ideas for that too.

[Basiliscus]

12 hours ago, Jim said:

This is exactly why I created this forum. A place for people doing interesting stuff...because we can  

Thanks for the MQTT pointer. These are the things that are really helping me out. I appreciate that a lot.

Please stick around  

In the future I would like to do face recognition with a camera on my robot. Maybe you have ideas for that too.

No worries!

I will stick around but I probably won't be in this forum every day so feel free to @ me if you want me to look at something (or send me a PM). I've followed you on Flickr and you're welcome to browse my GitHub - the "photos-categoriser" project is the one I was talking about earlier. I haven't done much on it in ages.

Will be interested to see how this project progresses!

[Jim]

6 hours ago, Basiliscus said:

No worries!

I will stick around but I probably won't be in this forum every day so feel free to @ me if you want me to look at something (or send me a PM). I've followed you on Flickr and you're welcome to browse my GitHub - the "photos-categoriser" project is the one I was talking about earlier. I haven't done much on it in ages.

Will be interested to see how this project progresses!

Cool. No worries about not being around every day. There are nine Technic sets underway for review, so I will take a detour (from Robotics) the next couple of weeks anyway. 

I will surely take a look at your Git repository and check out your project. And I know where to find you 

Edit: some interesting stuff on your Git page 

[Basiliscus]

16 hours ago, Jim said:

Cool. No worries about not being around every day. There are nine Technic sets underway for review, so I will take a detour (from Robotics) the next couple of weeks anyway. 

I will surely take a look at your Git repository and check out your project. And I know where to find you 

Edit: some interesting stuff on your Git page 

Thanks! I moved jobs in April 2018 and prior to that I made a bit of an effort to make some things available to show during interviews.

If you have a GitHub account feel free to follow me or share it here so I can see all the code.

Best of luck!

[Jim]

Just now, Basiliscus said:

Thanks! I moved jobs in April 2018 and prior to that I made a bit of an effort to make some things available to show during interviews.

If you have a GitHub account feel free to follow me or share it here so I can see all the code.

Best of luck!

Having a portfolio with interesting projects is always helpful. Wise choice.

I use Azure DevOps with Git. 

[Jim]

On 11/27/2018 at 9:33 PM, Basiliscus said:

Rather than using websockets you'd be better off using MQTT - set up a broker and make the web app send messages to the broker and the EV3 a subscriber. The architecture handles packet loss and general web instability a lot better and it's used a lot in IOT applications.

Have you used Mosquitto for MQTT? I have been playing around with it, but I can't get my first message accross.

O never mind....I just sent my first message. Not sure why it is working 

I was going through these instructions:

https://www.instructables.com/id/Installing-MQTT-BrokerMosquitto-on-Raspberry-Pi/

 

When I use this command:

mosquitto_pub -h test.mosquitto.org -t newtopic/test -m 'Hello'

I sometimes get the error that -m is an invalid option. And sometimes it is working. I had a feeling it has something to do with copy-paste from the interwebs, because the behavior is erratic. But now a command that previously failed is also working. Strange.

[Jim]

There is also https://pypi.org/project/paho-mqtt/. Not sure which one is better for my projects.

[Basiliscus]

Hmm seems strange that the error is occuring intermittently! That's the worst type of error to debug. To me I'd be pointing the finger at the broker in that case.

Where are you running the broker? In Azure? Despite recommending it I've never actually used MQTT - ironically given your skillset I've only used the less lightweight AMQP protocol via Event Hubs in Azure. I think RabbitMQ might be a shout if you are struggling with what you have now.

I'm off work until Wednesday but I could spare some time then to check for you? I don't have an EV3 module or a Raspberry Pi though so I'd be working blind.

Hope that helps!

[MajorAlvega]

test.mosquitto.org may fail sometimes, they don't offer SLA's. But I really never had problems, perhaps you are using topics with spaces and missing the quotes?

you can also use your own mosquitto broker (laptop or even ev3dev itself, is very light). I use on my laptop, this way don't need to ask my networking team to open MQTT port on the corporate firewall.

paho-mqtt is good if you want to use python for MQTT - I've been using mostly bash scripts so mosquitto_pub and mosquitto_sub are enough but when things get more complicated and bash isn't enough I use paho-mqtt (not very often... I'm still learning python).

[Basiliscus]

If @MajorAlvega is right and you're allowing the broker to be hosted somewhere else, I'd avoid that architecture and host the broker yourself. That way you'll be able to trace the routes yourself.

[Jim]

I was starting with my own broker on EV3, but I got some permission errors. So I changed to test.mosquitto.org which did work....sort of.

9 hours ago, MajorAlvega said:

test.mosquitto.org may fail sometimes, they don't offer SLA's. But I really never had problems, perhaps you are using topics with spaces and missing the quotes?

I tried every variant hehe.

But no worries guys. Yesterday was my first deep dive into MQTT, so I need to learn the ropes. I will do some more testing and share my experience.