Difference between revisions of "Telepresence"

From Fixme.ch
Jump to: navigation, search
 
(40 intermediate revisions by 3 users not shown)
Line 3: Line 3:
  
 
* (HD) Telepresence between FIXME and PTL
 
* (HD) Telepresence between FIXME and PTL
 +
[[File:Telepresence_FIXME.jpg|512px]]
  
−
== Obstacles ==
+
== Current issues ==
−
A list of potential obstacles
+
 
−
* The Raspberry Pi doesn't support high-rate webcam on its USB port
+
* The telepresence part dosen't work
−
* Raspberry Pi camera module delivery time
+
* OK Accelerated video decoding and display on the Raspi
 
* PTL outgoing bandwidth
 
* PTL outgoing bandwidth
 +
* Build a wooden window-like frame with curtains
 +
* gstreamer-omx crashes way too often!
 +
* We're now using [https://www.linux-projects.org/uv4l/ uv4l] instead of gstreamer and it looks quite promising!
  
−
== Component  ==
+
== Hardware ==
−
* 2 (4?) Raspberry PI
+
* 2 Raspberry PI
 
* 2 Camera modules
 
* 2 Camera modules
 
* Custom printed box
 
* Custom printed box
 +
 +
== Video streaming ==
 +
 +
== Archives ==
  
 
== Video Streaming ==
 
== Video Streaming ==
  
−
/etc/rc.local:
+
It works!
 +
 
 +
1. Install gstreamer with omx support
 +
 
 
<pre>
 
<pre>
−
su pi -c "raspivid -t 86400000 -w 1080 -h 720 -fps 30 -b 256000 -n -o - | cvlc -vvv stream:///dev/stdin --sout '#rtp{sdp=rtsp://:5000/}' :demux=h264" &
+
echo "deb http://vontaene.de/raspbian-updates/ . main" | sudo tee /etc/apt/sources.list.d/gstreamer.list
 +
sudo apt-get update
 +
sudo apt-get install libgstreamer1.0-0-dbg gstreamer1.0-tools libgstreamer-plugins-base1.0-0  gstreamer1.0-plugins-good gstreamer1.0-plugins-bad-dbg gstreamer1.0-omx gstreamer1.0-alsa
 
</pre>
 
</pre>
  
−
Client not yet working on the Raspi...
+
2. Install other required packages:
  
 
<pre>
 
<pre>
−
omxplayer rtsp://localhost:5000
+
sudo apt-get install vlc
 
</pre>
 
</pre>
  
−
However, this works quite well on a regular PC (video decoding done in software):
+
3. Configure /etc/rc.local to launch processes during startup:
 +
 
 +
=== RTSP ===
  
 
<pre>
 
<pre>
−
vlc rtsp://1.2.3.4:5000/
+
su pi -c "while :; do raspivid -t 86400000 -w 1280 -h 720 -fps 25 -b 256000 -n -o - | cvlc -vvv stream:///dev/stdin --sout '#rtp{sdp=rtsp://:1554/}' :demux=h264; sleep 60; done" &
 +
su pi -c "while :; do gst-launch-1.0 rtspsrc location=rtsp://ptl.telepresence.fixme.ch:1554/ ! rtph264depay ! h264parse ! omxh264dec ! autovideosink; sleep 60; done" &
 
</pre>
 
</pre>
 +
 +
PTL raspi must use rtsp://fixme.telepresence.fixme.ch:1554/ instead.
 +
 +
Issues:
 +
* NAT...
 +
 +
=== UDP ===
 +
 +
Issues:
 +
* Client *must* be started first because of caps transmission
 +
 +
== FIXME side ==
 +
 +
<pre>
 +
su pi -c "while :; do raspivid -t 999999 -w 1280 -h 720 -fps 25 -hf -b 1000000 -n -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! udpsink host=ptl.telepresence.fixme.ch port=5000 force-ipv4=true > /tmp/server.log 2>&1; sleep 60; done" &
 +
su pi -c "while :; do gst-launch-1.0 -v udpsrc port=5000 ! gdpdepay ! rtph264depay ! h264parse ! omxh264dec ! autovideosink > /tmp/client.log 2>&1; sleep 60; done" &
 +
</pre>
 +
 +
== PTL side ==
 +
 +
<pre>
 +
su pi -c "while :; do raspivid -t 999999 -w 1280 -h 720 -fps 25 -hf -b 1000000 -n -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! udpsink host=fixme.telepresence.fixme.ch port=5000 force-ipv4=true > /tmp/server.log 2>&1; sleep 60; done" &
 +
su pi -c "while :; do gst-launch-1.0 -v udpsrc port=5000 ! gdpdepay ! rtph264depay ! h264parse ! omxh264dec ! autovideosink > /tmp/client.log 2>&1; sleep 60; done" &
 +
</pre>
 +
 +
=== TCP ===
 +
 +
/etc/rc.local côté FIXME:
 +
 +
<pre>
 +
su pi -c "while :; do raspivid -t 86400000 -w 1280 -h 720 -fps 25 -b 512000 -n -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink host=0.0.0.0 port=5000 > /tmp/server.log 2>&1; sleep 60; done" &
 +
su pi -c "while :; do gst-launch-1.0 -v tcpclientsrc host=ptl.telepresence.fixme.ch port=1554 ! gdpdepay ! rtph264depay ! h264parse ! omxh264dec ! autovideosink sync=false > /tmp/client.log 2>&1; sleep 60; done" &
 +
</pre>
 +
 +
/etc/rc.local côté PTL:
 +
 +
(note: pour l'instant le port 5000 est utilisé a fixme pour contourner les problemes de PAT/NAT au ptl en utilisant le meme port 1554 dans les 2 sens)
 +
<pre>
 +
su pi -c "while :; do raspivid -t 86400000 -w 1280 -h 720 -fps 25 -b 256000 -vf -hf -n -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink host=0.0.0.0 port=1554; sleep 60; done" &
 +
su pi -c "while :; do gst-launch-1.0 -v tcpclientsrc host=fixme.telepresence.fixme.ch port=5000 ! gdpdepay ! rtph264depay ! h264parse ! omxh264dec ! autovideosink sync=false; sleep 60; done" &
 +
</pre>
 +
 +
Test depuis un PC:
 +
 +
<pre>
 +
gst-launch-1.0 -v tcpclientsrc host=YOUR-PI-IP-ADDRESS port=5000  ! gdpdepay !  rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false
 +
</pre>
 +
 +
== CEC - TV Screen control through HDMI ==
 +
 +
Reference: http://raspberrypi.stackexchange.com/questions/7054/cec-wake-up-command
 +
 +
<pre>
 +
git clone git://github.com/Pulse-Eight/libcec.git
 +
git cd libcec
 +
./configure --with-rpi-include-path=/opt/vc/include --with-rpi-lib-path=/opt/vc/lib --enable-rpi
 +
make
 +
sudo make install
 +
</pre>
 +
 +
Switch the TV screen on:
 +
 +
<pre>
 +
echo 'on 0' | /usr/local/bin/cec-client -s
 +
</pre>
 +
 +
Switch the TV screen off:
 +
 +
<pre>
 +
echo 'standby 0' | /usr/local/bin/cec-client -s
 +
</pre>
 +
 +
=== Server ===
 +
 +
<pre>
 +
$ sudo raspivid -p 0,-100,500,500 -t 0 -o - -w 1280 -h 720 -fps 25 -b 1000000 | cvlc -vvv stream:///dev/stdin --sout '#rtp{sdp=rtsp://:9000/}' :demux=h264
 +
</pre>
 +
 +
Do do not forget to firewall access to the service!
 +
 +
=== Client ===
 +
 +
<pre>
 +
$ omxplayer -o hdmi -s --live rtsp://1.2.3.4:9000/
 +
</pre>
 +
 +
== Network (@PTL) ==
 +
 +
<pre>
 +
iptables -t nat -A PREROUTING -p tcp --dport 5000 -j DNAT --to-destination raspi_internal_ip:5000
 +
</pre>
 +
<pre>
 +
#ACTION    SOURCE      DEST                                  PROTO        DEST PORT
 +
DNAT      net        loc:local-IP-address[:local-port]      protocol    port-number
 +
DNAT      net        loc:raspberrypi                        udp          5000
 +
</pre>
 +
 +
== Automation with Puppet ==
 +
 +
The basic idea is to develop are shared puppet recipe hosted in a git repository to allow multiple raspi managed by different entities to all share the same verified configuration. This will hopefully provide insurance that the system really works point-to-point and that there's no way to spy in-between.
 +
 +
=== Setup from scratch ===
 +
 +
<pre>
 +
$ whoami
 +
pi
 +
$ sudo apt-get install -y puppet
 +
$ git clone ssh://gitolite@git.fixme.ch:1337/telepresence # send your ssh public ssh to François
 +
$ sudo ~/telepresence/run.sh
 +
</pre>
 +
 +
=== Logs ===
 +
 +
All puppet run results are sent to a publicly accessible e-mail address.
 +
 +
* http://mailinator.com/inbox.jsp?to=fixme-ptl-telepresence
 +
 +
== Ideas ==
 +
 +
* http://jamesharrison.github.io/openob/intro.html#architecture
  
 
== Participants ==
 
== Participants ==
 
* [[User:Francois]]
 
* [[User:Francois]]
 +
* [[User:Freestrom]]
 +
* [[User:mic]]

Latest revision as of 23:50, 15 November 2017

Description

  • (HD) Telepresence between FIXME and PTL

Telepresence FIXME.jpg

Current issues

  • The telepresence part dosen't work
  • OK Accelerated video decoding and display on the Raspi
  • PTL outgoing bandwidth
  • Build a wooden window-like frame with curtains
  • gstreamer-omx crashes way too often!
  • We're now using uv4l instead of gstreamer and it looks quite promising!

Hardware

  • 2 Raspberry PI
  • 2 Camera modules
  • Custom printed box

Video streaming

Archives

Video Streaming

It works!

1. Install gstreamer with omx support 

echo "deb http://vontaene.de/raspbian-updates/ . main" | sudo tee /etc/apt/sources.list.d/gstreamer.list
sudo apt-get update
sudo apt-get install libgstreamer1.0-0-dbg gstreamer1.0-tools libgstreamer-plugins-base1.0-0   gstreamer1.0-plugins-good gstreamer1.0-plugins-bad-dbg gstreamer1.0-omx gstreamer1.0-alsa

2. Install other required packages: 

sudo apt-get install vlc

3. Configure /etc/rc.local to launch processes during startup:

RTSP

su pi -c "while :; do raspivid -t 86400000 -w 1280 -h 720 -fps 25 -b 256000 -n -o - | cvlc -vvv stream:///dev/stdin --sout '#rtp{sdp=rtsp://:1554/}' :demux=h264; sleep 60; done" &
su pi -c "while :; do gst-launch-1.0 rtspsrc location=rtsp://ptl.telepresence.fixme.ch:1554/ ! rtph264depay ! h264parse ! omxh264dec ! autovideosink; sleep 60; done" &

PTL raspi must use rtsp://fixme.telepresence.fixme.ch:1554/ instead.

Issues:

  • NAT...

UDP

Issues:

  • Client *must* be started first because of caps transmission

FIXME side

su pi -c "while :; do raspivid -t 999999 -w 1280 -h 720 -fps 25 -hf -b 1000000 -n -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! udpsink host=ptl.telepresence.fixme.ch port=5000 force-ipv4=true > /tmp/server.log 2>&1; sleep 60; done" &
su pi -c "while :; do gst-launch-1.0 -v udpsrc port=5000 ! gdpdepay ! rtph264depay ! h264parse ! omxh264dec ! autovideosink > /tmp/client.log 2>&1; sleep 60; done" &

PTL side

su pi -c "while :; do raspivid -t 999999 -w 1280 -h 720 -fps 25 -hf -b 1000000 -n -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! udpsink host=fixme.telepresence.fixme.ch port=5000 force-ipv4=true > /tmp/server.log 2>&1; sleep 60; done" &
su pi -c "while :; do gst-launch-1.0 -v udpsrc port=5000 ! gdpdepay ! rtph264depay ! h264parse ! omxh264dec ! autovideosink > /tmp/client.log 2>&1; sleep 60; done" &

TCP

/etc/rc.local côté FIXME: 

su pi -c "while :; do raspivid -t 86400000 -w 1280 -h 720 -fps 25 -b 512000 -n -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink host=0.0.0.0 port=5000 > /tmp/server.log 2>&1; sleep 60; done" &
su pi -c "while :; do gst-launch-1.0 -v tcpclientsrc host=ptl.telepresence.fixme.ch port=1554 ! gdpdepay ! rtph264depay ! h264parse ! omxh264dec ! autovideosink sync=false > /tmp/client.log 2>&1; sleep 60; done" &

/etc/rc.local côté PTL:

(note: pour l'instant le port 5000 est utilisé a fixme pour contourner les problemes de PAT/NAT au ptl en utilisant le meme port 1554 dans les 2 sens) 

su pi -c "while :; do raspivid -t 86400000 -w 1280 -h 720 -fps 25 -b 256000 -vf -hf -n -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink host=0.0.0.0 port=1554; sleep 60; done" &
su pi -c "while :; do gst-launch-1.0 -v tcpclientsrc host=fixme.telepresence.fixme.ch port=5000 ! gdpdepay ! rtph264depay ! h264parse ! omxh264dec ! autovideosink sync=false; sleep 60; done" &

Test depuis un PC: 

gst-launch-1.0 -v tcpclientsrc host=YOUR-PI-IP-ADDRESS port=5000  ! gdpdepay !  rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false

CEC - TV Screen control through HDMI

Reference: http://raspberrypi.stackexchange.com/questions/7054/cec-wake-up-command 

git clone git://github.com/Pulse-Eight/libcec.git
git cd libcec
./configure --with-rpi-include-path=/opt/vc/include --with-rpi-lib-path=/opt/vc/lib --enable-rpi
make
sudo make install

Switch the TV screen on: 

echo 'on 0' | /usr/local/bin/cec-client -s

Switch the TV screen off: 

echo 'standby 0' | /usr/local/bin/cec-client -s

Server

$ sudo raspivid -p 0,-100,500,500 -t 0 -o - -w 1280 -h 720 -fps 25 -b 1000000 | cvlc -vvv stream:///dev/stdin --sout '#rtp{sdp=rtsp://:9000/}' :demux=h264

Do do not forget to firewall access to the service!

Client

$ omxplayer -o hdmi -s --live rtsp://1.2.3.4:9000/

Network (@PTL)

iptables -t nat -A PREROUTING -p tcp --dport 5000 -j DNAT --to-destination raspi_internal_ip:5000

#ACTION    SOURCE      DEST                                   PROTO        DEST PORT
DNAT       net         loc:local-IP-address[:local-port]      protocol     port-number
DNAT       net         loc:raspberrypi                        udp          5000

Automation with Puppet

The basic idea is to develop are shared puppet recipe hosted in a git repository to allow multiple raspi managed by different entities to all share the same verified configuration. This will hopefully provide insurance that the system really works point-to-point and that there's no way to spy in-between.

Setup from scratch

$ whoami
pi
$ sudo apt-get install -y puppet
$ git clone ssh://gitolite@git.fixme.ch:1337/telepresence # send your ssh public ssh to François
$ sudo ~/telepresence/run.sh

Logs

All puppet run results are sent to a publicly accessible e-mail address.

Ideas

Participants

Retrieved from "http://fixme.ch/w/index.php?title=Telepresence&oldid=13557"