Remote control and monitoring for a robot

25/12/2024

Hi! I am an assistant professor Kenta URANO at Kawaguchi lab.

In our lab, we have multiple mobile robots and have been researching with them.

We use Autoware.universe for autonomous driving. However a robot has an onboard PC for Autoware and we cannot see the monitor of the PC while in autonomous driving. Moreover I have to grab a joypad (for remote control) with both hands and there's no way to grab an external monitor. I looked for a device like PlayStation Portal remote player and finally hit upon the following configuration.

wirelesssystem

The system consists of:

  • PC: here, an ordinally laptop
  • Hollyland Mars 400S Pro: ireless HDMI transmission device (here, only transmitter)
  • GAMESiR G8+: wireless game controller for smartphone
  • iPad mini: just iPad mini

and the connections are:

PC─Mars……(Wi-Fi)……iPad
└─……………(Bluetooth)……G8+

Hollyland Mars 400S Pro is a set of transmitter and receiver. In the configuration above, I do not use receiver because the transmitter has Wi-Fi function that allows smartphones and tables see the transmitted video with an app. Also, G8+ controller is connected to the PC, not iPad.

Here, we can see rviz (or any ubuntu screen content) and control the robot with controller.

rviz

Some notes;

No transmission distance test is conducted.なお， However there is no problem using this around 5 meters.

Mars 400S Pro accepts several power source like: NP-F battery, AC power adapter, USB-C. I recommend powering with USB-C from PC.

G8+ has several joypad modes such as DualShock4 or Xbox. As far as I test, DualShock4 mode has no problem on Ubuntu (I tested jstest-gtk and ROS2 joy node). Button assign of G8+ DualShock4 mode is same as normal DualShock4 or DualSense.

As smartphones and tables for monitoring, larger devices than iPad mini cannot be used because G8+ cannot hold such big device (I tried iPad Pro 11inch and no chance). However iPad mini is my recommendation because of the screen size.

Below is a test video for checking video transmission delay.

I am curious about other lab's robot monitoring. I will be happy if I can see best practice!