Updating image_streamer to work with JeVois camera
Starting with y2018 code and updating it to work with the new
JeVois camera. The JeVois camera sends a raw image. This
needs to be converted to work in a MJPEG stream.
March 9, 2019 (Tyler)
Responded to Brian's comments.
March 7, 2019 (Tyler)
Rebased on master.
March 6, 2019, 9:35 pm
A bit more cleanup. Ready for another review.
March 6, 2019, 7:10 pm
Moved files and updated scripts and build files to work.
Responded to Austin's Gerrit comments.
March 6, 2019, 7:55 pm
Made new directory with better name and move image_streamer files
into it.
March 5, 2019, 9:22 pm
Reworked deploy script. Need to respond to Austin's comments.
I will do this next.
March 3, 2019, 5:50 pm
Working on deploy script. Not ready to be reviewed.
March 2, 2019, 8:15 pm
Added command line option to set the IP address for the roboRIO.
March 2, 2019, 6:30 pm
Removed y2018 code dependancy.
March 1, 2019, 10:29 pm
Cleaned up the code and set up to run for 9971 robot.
March 1, 2019, 9:20 pm
This version works and boots on the ODROID. The logging was keeping
the program from starting.
March 1, 2019, 8:20 pm
Has debug lines and first version to work with the ODROID.
Jay and Bahar helped me with it.
Change-Id: I93511cb0397eb961d9691791b87b4148289e159b
diff --git a/y2019/BUILD b/y2019/BUILD
index 4f11c36..7c30dad 100644
--- a/y2019/BUILD
+++ b/y2019/BUILD
@@ -1,5 +1,6 @@
load("//frc971:downloader.bzl", "robot_downloader")
load("//aos/build:queues.bzl", "queue_library")
+load("@com_google_protobuf//:protobuf.bzl", "cc_proto_library")
robot_downloader(
start_binaries = [
@@ -140,6 +141,12 @@
visibility = ["//visibility:public"],
)
+cc_proto_library(
+ name = "vision_proto",
+ srcs = ["vision.proto"],
+ visibility = ["//visibility:public"],
+)
+
py_library(
name = "python_init",
srcs = ["__init__.py"],
diff --git a/y2019/image_streamer/BUILD b/y2019/image_streamer/BUILD
new file mode 100644
index 0000000..dc6c45f
--- /dev/null
+++ b/y2019/image_streamer/BUILD
@@ -0,0 +1,18 @@
+package(default_visibility = ["//visibility:public"])
+
+cc_binary(
+ name = "image_streamer",
+ srcs = ["image_streamer.cc"],
+ deps = [
+ "//aos/logging",
+ "//aos/logging:implementations",
+ "//aos/vision/blob:codec",
+ "//aos/vision/events:epoll_events",
+ "//aos/vision/events:socket_types",
+ "//aos/vision/events:udp",
+ "//aos/vision/image:image_stream",
+ "//aos/vision/image:reader",
+ "//y2019:vision_proto",
+ "@com_github_gflags_gflags//:gflags",
+ ],
+)
diff --git a/y2019/image_streamer/README_ODROID_setup.txt b/y2019/image_streamer/README_ODROID_setup.txt
new file mode 100644
index 0000000..16f7287
--- /dev/null
+++ b/y2019/image_streamer/README_ODROID_setup.txt
@@ -0,0 +1,120 @@
+# To build and deploy the image_streamer code to the ODROID, run
+# ./deploy.sh 10.9.71.179
+# This will also configure and copy a supervisor configuration
+# file, vision.conf, to /etc/supervisor/conf.d.
+
+# While the code can work with two cameras, as of March 4, 2019,
+# the robot only has one driver camera on it. To use
+# two cameras, set --single_camera to false. Use the
+# exposure flag to set the camera exposure.
+
+# To view a camera on a Debian laptop use:
+mplayer -tv device=/dev/video0 tv://
+# or if the video is not on video0 try
+mplayer -tv device=/dev/video4 tv://
+# Show available camera formats.
+# This came from https://superuser.com/questions/494575/ffmpeg-open-webcam-using-yuyv-but-i-want-mjpeg
+ffmpeg -f video4linux2 -list_formats all -i /dev/video2
+
+# Jevois notes:
+# To mount a jevois disk on a debian system, use y2019/vision/jevois-cmd
+cd y2019/vision/
+sudo ./jevois-cmd help
+sudo ./jevois-cmd usbsd # This will mount the disk.
+# If needed, install python-serial to run jevois-cmd
+sudo apt-get install python-serial
+
+# Austin debugged the camera startup problems with supervisorctl on the ODROID.
+# Here is a copy of how he did it. The problem was the logging
+# was not starting propery. Austin added a flag to image_streamer
+# so that it would not start logging by default.
+
+root@odroid:~# supervisorctl
+exposure_loop RUNNING pid 625, uptime 0:09:09
+vision FATAL Exited too quickly (process log may have details)
+supervisor> tail -1000 vision stderr
+SURE_ABSOLUTE from 297 to 300
+image_streamer(656)(00018): DEBUG at 0000000021.963290s: aos/vision/image/reader.cc: 175: SetCameraControl: Set camera control V4L2_CID_BRIGHTNESS from -3 to -40
+image_streamer(656)(00019): DEBUG at 0000000021.963585s: aos/vision/image/reader.cc: 175: SetCameraControl: Set camera control V4L2_CID_GAIN from 16 to 0
+image_streamer(656)(00020): INFO at 0000000021.965102s: aos/vision/image/reader.cc: 277: Init: framerate ended up at 1/30
+image_streamer(656)(00021): INFO at 0000000021.967263s: aos/vision/image/reader.cc: 59: Reader: Bat Vision Successfully Initialized.
+image_streamer(656)(00022): DEBUG at 0000000021.969020s: aos/vision/image/reader.cc: 292: Start: queueing buffers for the first time
+image_streamer(656)(00023): DEBUG at 0000000021.969275s: aos/vision/image/reader.cc: 301: Start: done with first queue
+image_streamer: y2019/vision/image_streamer.cc:60: BlobLog::BlobLog(const char *, const char *): Assertion `ofst_.is_open()' failed.
+
+supervisor> restart vision
+vision: ERROR (not running)
+vision: ERROR (spawn error)
+supervisor> status
+exposure_loop RUNNING pid 625, uptime 0:09:31
+vision STARTING
+supervisor> status
+exposure_loop RUNNING pid 625, uptime 0:09:33
+vision BACKOFF Exited too quickly (process log may have details)
+supervisor> status
+exposure_loop RUNNING pid 625, uptime 0:09:34
+vision BACKOFF Exited too quickly (process log may have details)
+supervisor> tail -1000 vision stderr
+SURE_ABSOLUTE from 297 to 300
+image_streamer(864)(00018): DEBUG at 0000000590.870582s: aos/vision/image/reader.cc: 175: SetCameraControl: Set camera control V4L2_CID_BRIGHTNESS from -3 to -40
+image_streamer(864)(00019): DEBUG at 0000000590.870856s: aos/vision/image/reader.cc: 175: SetCameraControl: Set camera control V4L2_CID_GAIN from 16 to 0
+image_streamer(864)(00020): INFO at 0000000590.872400s: aos/vision/image/reader.cc: 277: Init: framerate ended up at 1/30
+image_streamer(864)(00021): INFO at 0000000590.874543s: aos/vision/image/reader.cc: 59: Reader: Bat Vision Successfully Initialized.
+image_streamer(864)(00022): DEBUG at 0000000590.876289s: aos/vision/image/reader.cc: 292: Start: queueing buffers for the first time
+image_streamer(864)(00023): DEBUG at 0000000590.876547s: aos/vision/image/reader.cc: 301: Start: done with first queue
+image_streamer: y2019/vision/image_streamer.cc:60: BlobLog::BlobLog(const char *, const char *): Assertion `ofst_.is_open()' failed.
+
+supervisor> tail -1000 vision stdout
+SURE_ABSOLUTE from 297 to 300
+image_streamer(864)(00018): DEBUG at 0000000590.870582s: aos/vision/image/reader.cc: 175: SetCameraControl: Set camera control V4L2_CID_BRIGHTNESS from -3 to -40
+image_streamer(864)(00019): DEBUG at 0000000590.870856s: aos/vision/image/reader.cc: 175: SetCameraControl: Set camera control V4L2_CID_GAIN from 16 to 0
+image_streamer(864)(00020): INFO at 0000000590.872400s: aos/vision/image/reader.cc: 277: Init: framerate ended up at 1/30
+image_streamer(864)(00021): INFO at 0000000590.874543s: aos/vision/image/reader.cc: 59: Reader: Bat Vision Successfully Initialized.
+image_streamer(864)(00022): DEBUG at 0000000590.876289s: aos/vision/image/reader.cc: 292: Start: queueing buffers for the first time
+image_streamer(864)(00023): DEBUG at 0000000590.876547s: aos/vision/image/reader.cc: 301: Start: done with first queue
+image_streamer: y2019/vision/image_streamer.cc:60: BlobLog::BlobLog(const char *, const char *): Assertion `ofst_.is_open()' failed.
+
+root@odroid:~# /root/image_streamer --single_camera=true --camera0_exposure=600
+image_streamer(890)(00000): INFO at 0000000667.025668s: y2019/vision/image_streamer.cc: 298: main: In main.
+image_streamer(890)(00001): INFO at 0000000667.025722s: y2019/vision/image_streamer.cc: 300: main: before setting up tcp_server.
+image_streamer(890)(00002): INFO at 0000000667.025819s: aos/vision/events/tcp_server.cc: 76: SocketBindListenOnPort: connected to port: 80 on fd: 3
+image_streamer(890)(00003): INFO at 0000000667.025844s: y2019/vision/image_streamer.cc: 302: main: after setting up tcp_server.
+image_streamer(890)(00004): INFO at 0000000667.025872s: y2019/vision/image_streamer.cc: 303: main: In main.
+image_streamer(890)(00005): INFO at 0000000667.025896s: y2019/vision/image_streamer.cc: 305: main: In main.
+image_streamer(890)(00006): INFO at 0000000667.025913s: y2019/vision/image_streamer.cc: 307: main: In main.
+image_streamer(890)(00007): INFO at 0000000667.025933s: y2019/vision/image_streamer.cc: 309: main: In main.
+image_streamer(890)(00008): INFO at 0000000667.025952s: y2019/vision/image_streamer.cc: 311: main: In main.
+image_streamer(890)(00009): INFO at 0000000667.025968s: y2019/vision/image_streamer.cc: 314: main: In main.
+image_streamer(890)(00010): INFO at 0000000667.025985s: y2019/vision/image_streamer.cc: 317: main: In main.
+image_streamer(890)(00011): INFO at 0000000667.026001s: y2019/vision/image_streamer.cc: 319: main: In main.
+image_streamer(890)(00012): INFO at 0000000667.026017s: y2019/vision/image_streamer.cc: 322: main: In main.
+Before setting up udp socket 5001
+image_streamer(890)(00013): INFO at 0000000667.026090s: y2019/vision/image_streamer.cc: 324: main: In main.
+image_streamer(890)(00014): INFO at 0000000667.026142s: y2019/vision/image_streamer.cc: 328: main: In main.
+image_streamer(890)(00015): WARNING at 0000000667.026220s: aos/vision/image/reader.cc: 217: Init: xioctl VIDIOC_S_CROP due to 25 (Inappropriate ioctl for device)
+image_streamer(890)(00016): DEBUG at 0000000667.026646s: aos/vision/image/reader.cc: 162: SetCameraControl: Camera control V4L2_CID_EXPOSURE_AUTO was already 1
+image_streamer(890)(00017): DEBUG at 0000000667.027819s: aos/vision/image/reader.cc: 175: SetCameraControl: Set camera control V4L2_CID_EXPOSURE_ABSOLUTE from 300 to 600
+image_streamer(890)(00018): DEBUG at 0000000667.027887s: aos/vision/image/reader.cc: 175: SetCameraControl: Set camera control V4L2_CID_BRIGHTNESS from -3 to -40
+image_streamer(890)(00019): DEBUG at 0000000667.027956s: aos/vision/image/reader.cc: 175: SetCameraControl: Set camera control V4L2_CID_GAIN from 16 to 0
+image_streamer(890)(00020): INFO at 0000000667.029905s: aos/vision/image/reader.cc: 277: Init: framerate ended up at 1/30
+image_streamer(890)(00021): INFO at 0000000667.031824s: aos/vision/image/reader.cc: 59: Reader: Bat Vision Successfully Initialized.
+image_streamer(890)(00022): DEBUG at 0000000667.033340s: aos/vision/image/reader.cc: 292: Start: queueing buffers for the first time
+image_streamer(890)(00023): DEBUG at 0000000667.033369s: aos/vision/image/reader.cc: 301: Start: done with first queue
+Logging to file (./logging/blob_record_38.dat)
+Running Camera
+image_streamer(890)(00024): DEBUG at 0000000667.236660s: aos/vision/image/reader.cc: 162: SetCameraControl: Camera control V4L2_CID_EXPOSURE_AUTO was already 1
+image_streamer(890)(00025): DEBUG at 0000000667.238058s: aos/vision/image/reader.cc: 175: SetCameraControl: Set camera control V4L2_CID_EXPOSURE_ABSOLUTE from 15 to 600
+image_streamer(890)(00026): INFO at 0000000667.238178s: y2019/vision/image_streamer.cc: 278: ProcessImage: Data has length 29161
+image_streamer(890)(00027): INFO at 0000000667.238205s: y2019/vision/image_streamer.cc: 335: operator(): Got a frame cam0
+image_streamer(890)(00028): INFO at 0000000667.252786s: y2019/vision/image_streamer.cc: 278: ProcessImage: Data has length 29521
+image_streamer(890)(00029): INFO at 0000000667.252809s: y2019/vision/image_streamer.cc: 335: operator(): Got a frame cam0
+image_streamer(890)(00030): INFO at 0000000667.272826s: y2019/vision/image_streamer.cc: 278: ProcessImage: Data has length 29559
+image_streamer(890)(00031): INFO at 0000000667.272848s: y2019/vision/image_streamer.cc: 335: operator(): Got a frame cam0
+image_streamer(890)(00032): INFO at 0000000667.316659s: y2019/vision/image_streamer.cc: 278: ProcessImage: Data has length 29640
+image_streamer(890)(00033): INFO at 0000000667.316680s: y2019/vision/image_streamer.cc: 335: operator(): Got a frame cam0
+image_streamer(890)(00034): INFO at 0000000667.377320s: y2019/vision/image_streamer.cc: 278: ProcessImage: Data has length 65435
+image_streamer(890)(00035): INFO at 0000000667.377346s: y2019/vision/image_streamer.cc: 335: operator(): Got a frame cam0
+image_streamer(890)(00036): INFO at 0000000667.412857s: y2019/vision/image_streamer.cc: 278: ProcessImage: Data has length 65651
+image_streamer(890)(00037): INFO at 0000000667.412945s: y2019/vision/image_streamer.cc: 335: operator(): Got a frame cam0
+image_streamer(890)(00038): INFO at 0000000667.444955s: y2019/vision/image_streamer.cc: 278: ProcessImage: Data has length 65648
+
diff --git a/y2019/image_streamer/deploy.sh b/y2019/image_streamer/deploy.sh
new file mode 100755
index 0000000..584d09e
--- /dev/null
+++ b/y2019/image_streamer/deploy.sh
@@ -0,0 +1,63 @@
+#!/bin/bash
+
+set -e
+
+echo ""
+echo "USAGE: $0 ODROID_ip_address"
+echo "Example: $0 10.9.71.179"
+echo "Example: $0 10.99.71.179"
+echo ""
+
+if [ $# != 1 ]
+ then
+ echo "Illegal number of parameters"
+ exit
+fi
+
+if [[ $1 == -*[hH]* ]]
+ then
+ exit
+fi
+
+# Get the script directory (from https://devhints.io/bash)
+DIR="${0%/*}"
+
+# Move into the script directory
+cd "${DIR}"
+echo "# Working in `pwd`"
+
+ODROID_IP_ADDRESS=$1
+ODROID="root@${ODROID_IP_ADDRESS}"
+# Get the IP address of the roboRIO from the ODROID IP address
+# This is needed to properly configure supervisorctl on the ODROID
+# for image_streamer to communicate with the roboRIO.
+ROBORIO=`echo ${ODROID_IP_ADDRESS} | sed 's/\.[0-9]*$/.2/'`
+
+echo "# Using ODORID ${ODROID}"
+echo "# Using roboRIO ${ROBORIO}"
+
+# This builds the ODROID image_streamer code.
+echo -e "\n# Building image_streamer"
+(
+set -x
+bazel build -c opt //y2019/image_streamer:image_streamer --cpu=armhf-debian
+)
+
+echo -e "\n# Copy files to ODROID"
+(
+set -x
+rsync -av --progress ../../bazel-bin/y2019/image_streamer/image_streamer "${ODROID}":.
+rsync -av --progress README_ODROID_setup.txt "${ODROID}":.
+rsync -av --progress vision.conf "${ODROID}":/etc/supervisor/conf.d/
+)
+
+echo "# Make sure supervisorctl has the correct IP address."
+(
+set -x
+ssh "${ODROID}" sed -i -e "'/image_streamer/ s/10.9.71.2/${ROBORIO}/'" /etc/supervisor/conf.d/vision.conf
+
+ssh "${ODROID}" sync
+)
+
+echo -e "\nCan't restart image_streamer with supervisorctl because the USB devices don't come up reliably..." >&2
+echo "Restart the ODROID now" >&2
diff --git a/y2019/image_streamer/image_streamer.cc b/y2019/image_streamer/image_streamer.cc
new file mode 100644
index 0000000..cd83a4e
--- /dev/null
+++ b/y2019/image_streamer/image_streamer.cc
@@ -0,0 +1,373 @@
+#include "aos/vision/image/image_stream.h"
+
+#include <sys/stat.h>
+#include <deque>
+#include <fstream>
+#include <string>
+
+#include "aos/logging/implementations.h"
+#include "aos/logging/logging.h"
+#include "aos/vision/blob/codec.h"
+#include "aos/vision/events/socket_types.h"
+#include "aos/vision/events/udp.h"
+#include "aos/vision/image/reader.h"
+#include "gflags/gflags.h"
+#include "y2019/vision.pb.h"
+
+using ::aos::events::DataSocket;
+using ::aos::events::RXUdpSocket;
+using ::aos::events::TCPServer;
+using ::aos::vision::DataRef;
+using ::aos::vision::Int32Codec;
+using ::aos::monotonic_clock;
+using ::y2019::VisionControl;
+
+DEFINE_string(roborio_ip, "10.9.71.2", "RoboRIO IP Address");
+DEFINE_string(log, "",
+ "If non-empty, log images to the specified prefix with the image "
+ "index appended to the filename");
+DEFINE_bool(single_camera, true, "If true, only use video0");
+DEFINE_int32(camera0_exposure, 600, "Exposure for video0");
+DEFINE_int32(camera1_exposure, 600, "Exposure for video1");
+
+aos::vision::DataRef mjpg_header =
+ "HTTP/1.0 200 OK\r\n"
+ "Server: YourServerName\r\n"
+ "Connection: close\r\n"
+ "Max-Age: 0\r\n"
+ "Expires: 0\r\n"
+ "Cache-Control: no-cache, private\r\n"
+ "Pragma: no-cache\r\n"
+ "Content-Type: multipart/x-mixed-replace; "
+ "boundary=--boundary\r\n\r\n";
+
+struct Frame {
+ std::string data;
+};
+
+inline bool FileExist(const std::string &name) {
+ struct stat buffer;
+ return (stat(name.c_str(), &buffer) == 0);
+}
+
+class BlobLog {
+ public:
+ explicit BlobLog(const char *prefix, const char *extension) {
+ int index = 0;
+ while (true) {
+ std::string file = prefix + std::to_string(index) + extension;
+ if (FileExist(file)) {
+ index++;
+ } else {
+ printf("Logging to file (%s)\n", file.c_str());
+ ofst_.open(file);
+ assert(ofst_.is_open());
+ break;
+ }
+ }
+ }
+
+ ~BlobLog() { ofst_.close(); }
+
+ void WriteLogEntry(DataRef data) { ofst_.write(&data[0], data.size()); }
+
+ private:
+ std::ofstream ofst_;
+};
+
+class UdpClient : public ::aos::events::EpollEvent {
+ public:
+ UdpClient(int port, ::std::function<void(void *, size_t)> callback)
+ : ::aos::events::EpollEvent(RXUdpSocket::SocketBindListenOnPort(port)),
+ callback_(callback) {}
+
+ private:
+ ::std::function<void(void *, size_t)> callback_;
+
+ void ReadEvent() override {
+ char data[1024];
+ size_t received_data_size = Recv(data, sizeof(data));
+ callback_(data, received_data_size);
+ }
+
+ size_t Recv(void *data, int size) {
+ return PCHECK(recv(fd(), static_cast<char *>(data), size, 0));
+ }
+};
+
+// TODO(aschuh & michael) Pull this out.
+template <typename PB>
+class ProtoUdpClient : public UdpClient {
+ public:
+ ProtoUdpClient(int port, ::std::function<void(const PB &)> proto_callback)
+ : UdpClient(port, ::std::bind(&ProtoUdpClient::ReadData, this,
+ ::std::placeholders::_1,
+ ::std::placeholders::_2)),
+ proto_callback_(proto_callback) {}
+
+ private:
+ ::std::function<void(const PB &)> proto_callback_;
+
+ void ReadData(void *data, size_t size) {
+ PB pb;
+ pb.ParseFromArray(data, size);
+ proto_callback_(pb);
+ }
+};
+
+class MjpegDataSocket : public aos::events::SocketConnection {
+ public:
+ MjpegDataSocket(aos::events::TCPServerBase *server, int fd)
+ : aos::events::SocketConnection(server, fd) {
+ SetEvents(EPOLLOUT | EPOLLET);
+ }
+
+ ~MjpegDataSocket() { printf("Closed connection on descriptor %d\n", fd()); }
+
+ void DirectEvent(uint32_t events) override {
+ if (events & EPOLLOUT) {
+ NewDataToSend();
+ events &= ~EPOLLOUT;
+ }
+ // Other end hung up. Ditch the connection.
+ if (events & EPOLLHUP) {
+ CloseConnection();
+ events &= ~EPOLLHUP;
+ return;
+ }
+ if (events) {
+ aos::events::EpollEvent::DirectEvent(events);
+ }
+ }
+
+ void ReadEvent() override {
+ // Ignore reads, but don't leave them pending.
+ ssize_t count;
+ char buf[512];
+ while (true) {
+ count = read(fd(), &buf, sizeof buf);
+ if (count <= 0) {
+ if (errno != EAGAIN) {
+ CloseConnection();
+ return;
+ }
+ break;
+ } else if (!ready_to_receive_) {
+ // This 4 should match "\r\n\r\n".length();
+ if (match_i_ >= 4) {
+ printf("reading after last match\n");
+ continue;
+ }
+ for (char c : aos::vision::DataRef(&buf[0], count)) {
+ if (c == "\r\n\r\n"[match_i_]) {
+ ++match_i_;
+ if (match_i_ >= 4) {
+ if (!ready_to_receive_) {
+ ready_to_receive_ = true;
+ RasterHeader();
+ }
+ }
+ } else if (match_i_ != 0) {
+ if (c == '\r') match_i_ = 1;
+ }
+ }
+ }
+ }
+ }
+
+ void RasterHeader() {
+ output_buffer_.push_back(mjpg_header);
+ NewDataToSend();
+ }
+
+ void RasterFrame(std::shared_ptr<Frame> frame) {
+ if (!output_buffer_.empty() || !ready_to_receive_) return;
+ sending_frame_ = frame;
+ aos::vision::DataRef data = frame->data;
+
+ size_t n_written = snprintf(data_header_tmp_, sizeof(data_header_tmp_),
+ "--boundary\r\n"
+ "Content-type: image/jpg\r\n"
+ "Content-Length: %zu\r\n\r\n",
+ data.size());
+ // This should never happen because the buffer should be properly sized.
+ if (n_written == sizeof(data_header_tmp_)) {
+ fprintf(stderr, "wrong sized buffer\n");
+ exit(-1);
+ }
+ LOG(INFO, "Frame size in bytes: data.size() = %zu\n",data.size());
+ output_buffer_.push_back(aos::vision::DataRef(data_header_tmp_, n_written));
+ output_buffer_.push_back(data);
+ output_buffer_.push_back("\r\n\r\n");
+ NewDataToSend();
+ }
+
+ void NewFrame(std::shared_ptr<Frame> frame) { RasterFrame(std::move(frame)); }
+
+ void NewDataToSend() {
+ while (!output_buffer_.empty()) {
+ auto &data = *output_buffer_.begin();
+
+ while (!data.empty()) {
+ int len = send(fd(), data.data(), data.size(), MSG_NOSIGNAL);
+ if (len == -1) {
+ if (errno == EAGAIN) {
+ // Next thinggy will pick this up.
+ return;
+ } else {
+ CloseConnection();
+ return;
+ }
+ } else {
+ data.remove_prefix(len);
+ }
+ }
+ output_buffer_.pop_front();
+ }
+ }
+
+ private:
+ char data_header_tmp_[512];
+ std::shared_ptr<Frame> sending_frame_;
+ std::deque<aos::vision::DataRef> output_buffer_;
+
+ bool ready_to_receive_ = false;
+ void CloseConnection() {
+ loop()->Delete(this);
+ close(fd());
+ delete this;
+ }
+ size_t match_i_ = 0;
+};
+
+class CameraStream : public ::aos::vision::ImageStreamEvent {
+ public:
+ CameraStream(::aos::vision::CameraParams params, const ::std::string &fname,
+ TCPServer<MjpegDataSocket> *tcp_server, bool log,
+ ::std::function<void()> frame_callback)
+ : ImageStreamEvent(fname, params),
+ tcp_server_(tcp_server),
+ frame_callback_(frame_callback) {
+ if (log) {
+ log_.reset(new BlobLog(FLAGS_log.c_str(), ".dat"));
+ }
+ }
+
+ void set_active(bool active) { active_ = active; }
+
+ bool active() const { return active_; }
+
+ void ProcessImage(DataRef data,
+ monotonic_clock::time_point /*monotonic_now*/) {
+ ++sampling;
+ // 20 is the sampling rate.
+ if (sampling == 20) {
+ int tmp_size = data.size() + sizeof(int32_t);
+ char *buf;
+ std::string log_record;
+ log_record.resize(tmp_size, 0);
+ {
+ buf = Int32Codec::Write(&log_record[0], tmp_size);
+ data.copy(buf, data.size());
+ }
+ if (log_) {
+ log_->WriteLogEntry(log_record);
+ }
+ sampling = 0;
+ }
+
+ if (active_) {
+ auto frame = std::make_shared<Frame>(Frame{std::string(data)});
+ tcp_server_->Broadcast(
+ [frame](MjpegDataSocket *event) { event->NewFrame(frame); });
+ }
+ frame_callback_();
+ }
+
+ private:
+ int sampling = 0;
+ TCPServer<MjpegDataSocket> *tcp_server_;
+ ::std::unique_ptr<BlobLog> log_;
+ ::std::function<void()> frame_callback_;
+ bool active_ = false;
+};
+
+int main(int argc, char **argv) {
+ gflags::ParseCommandLineFlags(&argc, &argv, false);
+ ::aos::logging::Init();
+ ::aos::logging::AddImplementation(
+ new ::aos::logging::StreamLogImplementation(stderr));
+ TCPServer<MjpegDataSocket> tcp_server_(80);
+ aos::vision::CameraParams params0;
+ params0.set_exposure(FLAGS_camera0_exposure);
+ params0.set_brightness(-40);
+ params0.set_width(320);
+ // params0.set_fps(10);
+ params0.set_height(240);
+
+ aos::vision::CameraParams params1 = params0;
+ params1.set_exposure(FLAGS_camera1_exposure);
+
+ ::y2019::VisionStatus vision_status;
+ LOG(INFO,
+ "The UDP socket should be on port 5001 to 10.9.71.2 for "
+ "the competition robot.\n");
+ LOG(INFO, "Starting UDP socket on port 5001 to %s\n",
+ FLAGS_roborio_ip.c_str());
+ ::aos::events::ProtoTXUdpSocket<::y2019::VisionStatus> status_socket(
+ FLAGS_roborio_ip.c_str(), 5001);
+
+ ::std::unique_ptr<CameraStream> camera1;
+ ::std::unique_ptr<CameraStream> camera0(new CameraStream(
+ params0, "/dev/video0", &tcp_server_, !FLAGS_log.empty(),
+ [&camera0, &camera1, &status_socket, &vision_status]() {
+ vision_status.set_low_frame_count(vision_status.low_frame_count() + 1);
+ LOG(INFO, "Got a frame cam0\n");
+ if (camera0->active()) {
+ status_socket.Send(vision_status);
+ }
+ }));
+ if (!FLAGS_single_camera) {
+ camera1.reset(new CameraStream(
+ params1, "/dev/video1", &tcp_server_, false,
+ [&camera0, &camera1, &status_socket, &vision_status]() {
+ vision_status.set_high_frame_count(vision_status.high_frame_count() +
+ 1);
+ LOG(INFO, "Got a frame cam1\n");
+ if (camera1->active()) {
+ status_socket.Send(vision_status);
+ }
+ }));
+ }
+
+ ProtoUdpClient<VisionControl> udp_client(
+ 5000, [&camera0, &camera1](const VisionControl &vision_control) {
+ bool cam0_active = false;
+ if (camera1) {
+ cam0_active = !vision_control.high_video();
+ camera0->set_active(!vision_control.high_video());
+ camera1->set_active(vision_control.high_video());
+ } else {
+ cam0_active = true;
+ camera0->set_active(true);
+ }
+ LOG(INFO, "Got control packet, cam%d active\n", cam0_active ? 0 : 1);
+ });
+
+ // Default to camera0
+ camera0->set_active(true);
+ if (camera1) {
+ camera1->set_active(false);
+ }
+
+ aos::events::EpollLoop loop;
+ loop.Add(&tcp_server_);
+ loop.Add(camera0.get());
+ if (camera1) {
+ loop.Add(camera1.get());
+ }
+ loop.Add(&udp_client);
+
+ printf("Running Camera\n");
+ loop.Run();
+}
diff --git a/y2019/image_streamer/videomappings.cfg b/y2019/image_streamer/videomappings.cfg
new file mode 100644
index 0000000..0c6c91b
--- /dev/null
+++ b/y2019/image_streamer/videomappings.cfg
@@ -0,0 +1,544 @@
+############ 971 Spartan Robotics
+#
+# This JeVois configuration file works with the 2019 image_streamer program
+# on the ODROID that is used to send images back to the driverstation. Each uncomented
+# line in this file is a camera configuration. The JeVois camaera can have
+# many configurations. I think it can have as many as 50. The default configuration
+# has a '*' at the end of the line. To avoid confusion and make it clear which
+# configuration to use, only one configuration is active in this file.
+#
+# This file resides on the JeVois in /JEVOIS/config/videomappings.cfg
+# It can be editted by mounting the microSD card on a computer and editting it.
+# It can also be editted by mounting the JeVois disk under Debian. This
+# is done by running
+# sudo y2019/vision/tools/jevois-cmd usbsd
+# This mounts the disk in /media/${USER}/JEVOIS
+#
+# Michael, Bahar, and Jay. March 1, 2019.
+#
+#
+######################################################################################################################
+#
+# JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
+# California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
+#
+# This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
+# redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
+# Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
+# without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
+# License for more details. You should have received a copy of the GNU General Public License along with this program;
+# if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+#
+# Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
+# Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
+######################################################################################################################
+#
+# JeVois smart camera operation modes and video mappings
+#
+# Format is: <USBmode> <USBwidth> <USBheight> <USBfps> <CAMmode> <CAMwidth> <CAMheight> <CAMfps> <Vendor> <Module> [*]
+#
+# CamMode can be only one of: YUYV, BAYER, RGB565
+# USBmode can be only one of: YUYV, GREY, MJPG, BAYER, RGB565, BGR24, NONE
+
+# USB to camera mode mappings (when USBmode is not NONE) are organized according to output format, which should be
+# unique (no two entries in this file should have same USBmode/USBwidth/USBheight/USBfps). Indeed, these modes can only
+# be selected by the host computer's video grabbing software, and they are selected by picking an output format in that
+# software. These modes cannot be chosen by the JeVois system itself. For these modes, the Module's process(inframe,
+# outframe) function will be called on every frame. Beware that Macs will not accept modes for which USBwidth is not a
+# multiple of 16.
+
+# Camera-only modes (when USBmode is NONE) mode mappings have no video output over USB, and are selected by interacting
+# with the JeVois hardware over serial ports. When USBmode is NONE, USBwidth, USBHeight, and USBfps are ignored and
+# should be set to 0 here. For these modes, the Module's process(inframe) function will be called on every frame. These
+# modes are usually the ones you would use when interfacing the JeVois camera to an Arduino or similar system that
+# cannot stream video over USB and will just receive data from the JeVois camera over a serial port.
+
+# The optional * at the end of one line indicates the format that should be the default one announced by the device to
+# the USB host. This is the one that most webcam programs will select by default when you start them. Note that the
+# guvcview program on linux seems to ignore this and to instead select the last mode you had selected the last time you
+# used the camera. This * cannot be on a mapping that has NONE USBmode. There should be only one * in the whole file.
+
+# Model JeVois-A33 camera sensor supported resolutions and frame rates:
+#
+# SXGA (1280 x 1024): up to 15 fps
+# VGA ( 640 x 480): up to 30 fps
+# CIF ( 352 x 288): up to 60 fps
+# QVGA ( 320 x 240): up to 60 fps
+# QCIF ( 176 x 144): up to 120 fps
+# QQVGA ( 160 x 120): up to 60 fps
+# QQCIF ( 88 x 72): up to 120 fps
+
+# Frame rates can be set to any value from 0.1fps to the maximum supported for the selected resolution. This is very
+# useful to avoid dropping frames. For example if you have an algorithm that runs at 26.3fps after all possible
+# optimizations, you can set the camera (and usb) frame rate to 26.3 and you will not drop frames (unless your algorithm
+# momentarily performs slower, hence adding a small margin may be a good idea, e.g., select 26.1fps camera and usb
+# rates). This is better than setting the frame rate to 30.0 as this would mean that every so often you would miss the
+# next camera frame and then have to wait for the next one to be captured. If your algorithm really runs at 26.3fps but
+# you specify 30.0fps camera frame rate, then the frames will actually end up being pumped to USB at only 15.0fps (i.e.,
+# by the time you finish processing the current frame, you have missed the next one from the camera, and you need to
+# wait for the following one).
+
+# Note on USB transfer rate: the maximum actual pixel data transfer rate is 3070*8000 = 23.9 Mbytes/s (which is 3kb/USB
+# microframe, max "high bandwidth" setting). Although USB 2.0 has a maximum theoretical rate of 480 Mbit/s, this
+# includes protocol overhead and not all of the bandwidth is available for isochronous (real-time stream) transfers,
+# which we use. This means that SXGA YUYV (2 bytes/pixel) can only transfer at a max rate of ~9.3 fps over the USB
+# link, although the camera can grab SXGA YUYV at 15 fps. SXGA in Bayer can achieve 15 fps transfer over USB since it
+# only uses 1 byte/pixel.
+
+# To test various video formats on a Linux host, the best is usually to use guvcview. However, this has two issues: 1)
+# it adds some formats which we do not natively support, like RGB3, YU12, YV12, etc, probably by doing some pixel
+# conversion internally over the actual supported modes; 2) support for RGB565 seems brittle, guvcview often crashes
+# when changing resolutions in RGB565 (called RGBP for RGB Packed).
+#
+# Hence, you may want to also try ffplay from the ffmpeg project, which can display all our supported modes and will
+# reject a mode if it does not exactly match what the hardware supports. Example:
+#
+# ffplay /dev/video0 -pixel_format yuyv422 -video_size 640x480
+#
+# The pixel_format values are: 'yuyv422' for YUYV, 'gray' for GRAY, 'rgb565' for RGB565, 'mjpeg' for MJPG, 'bgr24' for
+# BGR24, and 'bayer_rggb8' for BAYER. You can run 'ffplay -pix_fmts' to see the list of pixel formats that ffplay
+# supports.
+#
+# Here is another example where we record the output of JeVois to a file:
+#
+# ffmpeg -f v4l2 -pixel_format rgb565 -video_size 320x240 -framerate 22 -i /dev/video0 output.mp4
+
+# On Mac OSX, we recommend using the CamTwist app, as described in the JeVois documentation. You can also use ffplay for
+# OSX: Download the pre-compiled ffplay binary from the ffmpeg web site, and then run:
+#
+# ~/bin/ffplay -f avfoundation -i "JeVois" -video_size 640x300 -framerate 60 -pixel_format yuyv422
+#
+# (assuming you saved ffplay into your ~/bin/ directory).
+
+# Mac compatibility notes: The JeVois smart camera is correctly detected on Macs and works with PhotoBooth as long as:
+# 1) you have a mapping that outputs YUYV 640x480 (this is the one that PhotoBooth will select (at least on recent OSX
+# like El Capitan, etc); beware that it will also flip the image horizontally); 2) you have no active (not
+# commented-out) mapping with BAYER, RGB565, or BGR24 output. If you have any un-commented mapping with BAYER, RGB565,
+# or BGR24 in your videomappings.cfg, your JeVois smart camera will still be detected by your Mac, PhotoBooth will start
+# and try to use the camera, but it will only display a black screen. Our guess is that this is a bug in the Mac camera
+# driver. It is ok to have additional mappings with YUYV output, as well as mappings with MJPG or GREY output.
+
+#####################################################################################################
+#### Pass-through and simple pixel format conversion modes:
+#####################################################################################################
+#
+##YUYV 1280 960 15.0 BAYER 1280 960 15.0 JeVois Convert
+##YUYV 1280 720 15.0 BAYER 1280 720 15.0 JeVois Convert
+##YUYV 640 480 30.0 BAYER 640 480 30.0 JeVois Convert
+##YUYV 640 360 30.0 BAYER 640 360 30.0 JeVois Convert
+##YUYV 320 240 60.0 BAYER 320 240 60.0 JeVois Convert
+##YUYV 320 180 60.0 BAYER 320 180 60.0 JeVois Convert
+##YUYV 160 120 60.0 BAYER 160 120 60.0 JeVois Convert
+##YUYV 160 90 60.0 BAYER 160 90 60.0 JeVois Convert
+#
+##BAYER 1280 960 15.0 BAYER 1280 960 15.0 JeVois PassThrough
+##BAYER 1280 720 15.0 BAYER 1280 720 15.0 JeVois PassThrough
+##BAYER 640 480 30.0 BAYER 640 480 30.0 JeVois PassThrough
+##BAYER 640 360 30.0 BAYER 640 360 30.0 JeVois PassThrough
+##BAYER 320 240 60.0 BAYER 320 240 60.0 JeVois PassThrough
+##BAYER 320 180 60.0 BAYER 320 180 60.0 JeVois PassThrough
+##BAYER 160 120 60.0 BAYER 160 120 60.0 JeVois PassThrough
+##BAYER 160 90 60.0 BAYER 160 90 60.0 JeVois PassThrough
+#
+##BAYER 640 480 26.8 YUYV 640 480 26.8 JeVois Convert
+##BGR24 640 480 26.8 YUYV 640 480 26.8 JeVois Convert
+##GREY 640 480 26.8 YUYV 640 480 26.8 JeVois Convert
+##RGB565 640 480 26.8 YUYV 640 480 26.8 JeVois Convert
+#
+##MJPG 640 480 20.0 YUYV 640 480 20.0 JeVois Convert
+##MJPG 352 288 60.0 BAYER 352 288 60.0 JeVois Convert
+##MJPG 320 240 30.0 RGB565 320 240 30.0 JeVois Convert
+##MJPG 320 240 15.0 YUYV 320 240 15.0 JeVois Convert
+##MJPG 320 240 60.0 RGB565 320 240 60.0 JeVois Convert
+##MJPG 176 144 120.0 BAYER 176 144 120.0 JeVois Convert
+##MJPG 160 120 60.0 YUYV 160 120 60.0 JeVois Convert
+##MJPG 88 72 120.0 RGB565 88 72 120.0 JeVois Convert
+#
+##BAYER 1280 1024 15.0 BAYER 1280 1024 15.0 JeVois PassThrough
+##BAYER 640 480 30.0 BAYER 640 480 30.0 JeVois PassThrough
+##BAYER 352 288 60.0 BAYER 352 288 60.0 JeVois PassThrough
+##BAYER 320 240 60.0 BAYER 320 240 60.0 JeVois PassThrough
+##BAYER 176 144 120.0 BAYER 176 144 120.0 JeVois PassThrough
+##BAYER 160 120 60.0 BAYER 160 120 60.0 JeVois PassThrough
+##BAYER 88 72 120.0 BAYER 88 72 120.0 JeVois PassThrough
+#
+##RGB565 1280 1024 15.0 RGB565 1280 1024 15.0 JeVois PassThrough
+##RGB565 640 480 30.0 RGB565 640 480 30.0 JeVois PassThrough
+##RGB565 320 240 60.0 RGB565 320 240 60.0 JeVois PassThrough
+##RGB565 176 144 120.0 RGB565 176 144 120.0 JeVois PassThrough
+##RGB565 160 120 60.0 RGB565 160 120 60.0 JeVois PassThrough
+##RGB565 88 72 120.0 RGB565 88 72 120.0 JeVois PassThrough
+#
+##YUYV 1280 1024 7.5 YUYV 1280 1024 7.5 JeVois PassThrough
+## Bahar and Michael uncommented the following line. 2/27/2019
+##YUYV 640 480 30.0 YUYV 640 480 30.0 JeVois PassThrough
+# BAYER looked good.
+#MJPG 640 480 30.0 BAYER 640 480 30.0 JeVois Convert *
+MJPG 640 480 30.0 RGB565 640 480 30.0 JeVois Convert *
+##YUYV 640 480 30.0 YUYV 640 480 30.0 JeVois SaveVideo
+##YUYV 640 480 19.6 YUYV 640 480 19.6 JeVois PassThrough
+##YUYV 640 480 12.0 YUYV 640 480 12.0 JeVois PassThrough
+##YUYV 640 480 8.3 YUYV 640 480 8.3 JeVois PassThrough
+##YUYV 640 480 7.5 YUYV 640 480 7.5 JeVois PassThrough
+##YUYV 640 480 5.5 YUYV 640 480 5.5 JeVois PassThrough
+#
+#YUYV 320 240 60.0 YUYV 320 240 60.0 JeVois SaveVideo
+##YUYV 320 240 30.0 YUYV 320 240 30.0 JeVois SaveVideo
+##YUYV 320 240 15.0 YUYV 320 240 15.0 JeVois SaveVideo
+#
+##YUYV 160 120 60.0 YUYV 160 120 60.0 JeVois SaveVideo
+##YUYV 160 120 30.0 YUYV 160 120 30.0 JeVois PassThrough
+#
+##YUYV 352 288 60.0 YUYV 352 288 60.0 JeVois SaveVideo
+##YUYV 352 288 30.0 YUYV 352 288 30.0 JeVois PassThrough
+#
+#YUYV 176 144 120.0 YUYV 176 144 120.0 JeVois SaveVideo
+##YUYV 176 144 60.0 YUYV 176 144 60.0 JeVois PassThrough
+##YUYV 176 144 30.0 YUYV 176 144 30.0 JeVois PassThrough
+#
+##YUYV 88 72 120.0 YUYV 88 72 120.0 JeVois SaveVideo
+##YUYV 88 72 60.0 YUYV 88 72 60.0 JeVois PassThrough
+##YUYV 88 72 30.0 YUYV 88 72 30.0 JeVois PassThrough
+#
+#####################################################################################################
+#### Save video to disk, no preview over USB
+#####################################################################################################
+#
+#NONE 0 0 0 YUYV 320 240 60.0 JeVois SaveVideo
+#NONE 0 0 0 YUYV 320 240 30.0 JeVois SaveVideo
+#NONE 0 0 0 YUYV 176 144 120.0 JeVois SaveVideo
+#
+#####################################################################################################
+#### Demo: Saliency + gist + face detection + object recognition
+#####################################################################################################
+#
+#YUYV 640 312 50.0 YUYV 320 240 50.0 JeVois DemoSalGistFaceObj
+#
+#####################################################################################################
+#### Demo: JeVois intro movie, then Saliency + gist + face detection + object recognition
+#####################################################################################################
+#
+#YUYV 640 360 30.0 YUYV 320 240 30.0 JeVois JeVoisIntro
+#YUYV 640 480 30.0 YUYV 320 240 30.0 JeVois JeVoisIntro
+#
+#####################################################################################################
+#### Demo: Saliency and gist
+#####################################################################################################
+#
+##YUYV 176 90 120.0 YUYV 88 72 120.0 JeVois DemoSaliency
+##YUYV 320 150 60.0 YUYV 160 120 60.0 JeVois DemoSaliency
+##YUYV 352 180 120.0 YUYV 176 144 120.0 JeVois DemoSaliency
+##YUYV 352 180 100.0 YUYV 176 144 100.0 JeVois DemoSaliency
+## Michael and Bahar removed the ' *' from the end of the following line. 2/27/2019
+#YUYV 640 300 60.0 YUYV 320 240 60.0 JeVois DemoSaliency
+##YUYV 704 360 30.0 YUYV 352 288 30.0 JeVois DemoSaliency
+##YUYV 1280 600 15.0 YUYV 640 480 15.0 JeVois DemoSaliency
+#
+#####################################################################################################
+#### Production: Saliency and gist
+#####################################################################################################
+#
+## saliency + feature maps + gist
+##GREY 120 25 60.0 YUYV 320 240 60.0 JeVois SaliencyGist
+#
+## saliency + feature maps
+##GREY 120 15 60.0 YUYV 320 240 60.0 JeVois SaliencyGist
+#
+## saliency + gist
+##GREY 20 73 60.0 YUYV 320 240 60.0 JeVois SaliencyGist
+#
+## saliency only
+##GREY 20 15 60.0 YUYV 320 240 60.0 JeVois SaliencyGist
+#
+## gist only
+##GREY 72 16 60.0 YUYV 320 240 60.0 JeVois SaliencyGist
+#
+#####################################################################################################
+#### Demo: Background subtraction
+#####################################################################################################
+#
+##YUYV 640 240 15.0 YUYV 320 240 15.0 JeVois DemoBackgroundSubtract
+#YUYV 320 120 30.0 YUYV 160 120 30.0 JeVois DemoBackgroundSubtract
+#
+#####################################################################################################
+#### Demo: QR-code and barcode detection and decoding
+#####################################################################################################
+#
+##YUYV 640 526 15.0 YUYV 640 480 15.0 JeVois DemoQRcode
+#YUYV 320 286 30.0 YUYV 320 240 30.0 JeVois DemoQRcode
+##NONE 0 0 0 YUYV 640 480 15.0 JeVois DemoQRcode
+##NONE 0 0 0 YUYV 320 240 30.0 JeVois DemoQRcode
+#
+#####################################################################################################
+#### Road following using vanishing point
+#####################################################################################################
+#
+#NONE 0 0 0 YUYV 320 240 30.0 JeVois RoadNavigation
+##NONE 0 0 0 YUYV 176 144 120.0 JeVois RoadNavigation
+#YUYV 320 256 30.0 YUYV 320 240 30.0 JeVois RoadNavigation
+##YUYV 176 160 120.0 YUYV 176 144 120.0 JeVois RoadNavigation
+#
+#####################################################################################################
+#### Demo of ARM-Neon SIMD image processing
+#####################################################################################################
+#
+#YUYV 960 240 30.0 YUYV 320 240 30.0 JeVois DemoNeon
+#
+#####################################################################################################
+#### Dense SIFT using VLfeat library
+#####################################################################################################
+#
+## very slow, min keypoint step is 17
+##YUYV 448 240 5.0 YUYV 320 240 5.0 JeVois DenseSift
+#
+## slow too, min keypoint step is 11
+##YUYV 288 120 5.0 YUYV 160 120 5.0 JeVois DenseSift
+#
+## raw keypoints only, assuming step=11, binsize=8
+#GREY 128 117 5.0 YUYV 160 120 5.0 JeVois DenseSift
+#
+#####################################################################################################
+#### Salient regions
+#####################################################################################################
+#
+#YUYV 64 192 25.0 YUYV 320 240 25.0 JeVois SalientRegions
+##YUYV 100 400 10.0 YUYV 640 480 10.0 JeVois SalientRegions
+#
+#####################################################################################################
+#### Superpixel image segmentation/clustering
+#####################################################################################################
+#
+#GREY 320 240 30.0 YUYV 320 240 30.0 JeVois SuperPixelSeg
+#
+#####################################################################################################
+#### Eye tracking using the openEyes toolkit
+#####################################################################################################
+#
+##GREY 640 480 30.0 YUYV 640 480 30.0 JeVois DemoEyeTracker
+##GREY 320 240 60.0 YUYV 320 240 60.0 JeVois DemoEyeTracker
+#GREY 176 144 120.0 YUYV 176 144 120.0 JeVois DemoEyeTracker
+#
+#####################################################################################################
+#### Demo: ArUco augmented-reality markers detection and decoding
+#####################################################################################################
+#
+#NONE 0 0 0.0 YUYV 320 240 30.0 JeVois DemoArUco
+#YUYV 320 260 30.0 YUYV 320 240 30.0 JeVois DemoArUco
+#YUYV 640 500 20.0 YUYV 640 480 20.0 JeVois DemoArUco
+#
+#####################################################################################################
+#### Edge detection using Canny
+#####################################################################################################
+#
+#GREY 640 480 29.0 YUYV 640 480 29.0 JeVois EdgeDetection
+#GREY 320 240 59.0 YUYV 320 240 59.0 JeVois EdgeDetection
+##GREY 176 144 119.0 YUYV 176 144 119.0 JeVois EdgeDetection
+#
+#####################################################################################################
+#### Edge detection using 4 Canny filters in parallel, with different settings
+#####################################################################################################
+#
+#GREY 320 960 45.0 YUYV 320 240 45.0 JeVois EdgeDetectionX4
+#
+#####################################################################################################
+#### Color-based object tracker
+#####################################################################################################
+#
+#NONE 0 0 0.0 YUYV 320 240 60.0 JeVois ObjectTracker
+#YUYV 320 254 60.0 YUYV 320 240 60.0 JeVois ObjectTracker
+#
+#####################################################################################################
+#### GPU color image processing demo
+#####################################################################################################
+#
+##RGB565 320 240 22.0 YUYV 320 240 22.0 JeVois DemoGPU
+#
+#####################################################################################################
+#### Combo CPU multithreaded saliency/gist + 4x GPU grayscale image processing demo
+#####################################################################################################
+#
+#GREY 160 495 60.0 YUYV 160 120 60.0 JeVois DemoCPUGPU
+#
+#####################################################################################################
+#### Fast optical flow computation
+#####################################################################################################
+#
+#GREY 176 288 100 YUYV 176 144 100 JeVois OpticalFlow
+#
+#####################################################################################################
+#### Object detection using SURF keypoints
+#####################################################################################################
+#
+#YUYV 320 252 30.0 YUYV 320 240 30.0 JeVois ObjectDetect
+#
+#####################################################################################################
+#### Salient region detection and identification using SURF keypoints
+#####################################################################################################
+#
+#YUYV 320 288 30.0 YUYV 320 240 30.0 JeVois SaliencySURF
+#
+#####################################################################################################
+#### CPU + GPU + NEON burn test
+#####################################################################################################
+#
+##YUYV 640 300 10.0 YUYV 320 240 10.0 JeVois BurnTest
+#
+#####################################################################################################
+#### Python tests
+#####################################################################################################
+#
+#YUYV 640 480 15.0 YUYV 640 480 15.0 JeVois PythonTest
+#GREY 640 480 20.0 YUYV 640 480 20.0 JeVois PythonOpenCV
+#YUYV 352 288 30.0 YUYV 352 288 30.0 JeVois PythonSandbox
+#
+#####################################################################################################
+#### Image color filtering
+#####################################################################################################
+#
+#YUYV 640 240 30.0 YUYV 320 240 30.0 JeVois ColorFiltering
+#
+#####################################################################################################
+#### Dice counting tutorial
+#####################################################################################################
+#
+#YUYV 640 480 7.5 YUYV 640 480 7.5 JeVois DiceCounter
+#
+#####################################################################################################
+#### Augmented reality markers with ARtoolkit
+#####################################################################################################
+#
+#NONE 0 0 0 YUYV 320 240 60.0 JeVois DemoARtoolkit
+#NONE 0 0 0 YUYV 640 480 30.0 JeVois DemoARtoolkit
+#NONE 0 0 0 YUYV 1280 1024 15.0 JeVois DemoARtoolkit
+#YUYV 320 258 60.0 YUYV 320 240 60.0 JeVois DemoARtoolkit
+##YUYV 640 498 30.0 YUYV 640 480 30.0 JeVois DemoARtoolkit
+#
+#####################################################################################################
+#### Augmented reality markers with ARtoolkit, ArUco, and QR-Code
+#####################################################################################################
+#
+##YUYV 320 306 50.0 YUYV 320 240 50.0 JeVois MarkersCombo
+#YUYV 640 546 20.0 YUYV 640 480 20.0 JeVois MarkersCombo
+#
+#####################################################################################################
+#### Detect objects in scenes using darknet YOLO deep neural network
+#####################################################################################################
+#
+#YUYV 1280 480 15.0 YUYV 640 480 15.0 JeVois DarknetYOLO
+#
+#####################################################################################################
+#### Detect objects in scenes using darknet deep neural network
+#####################################################################################################
+#
+#YUYV 544 240 15.0 YUYV 320 240 15.0 JeVois DarknetSingle
+##YUYV 448 240 15.0 YUYV 320 240 15.0 JeVois DarknetSingle
+##YUYV 864 480 15.0 YUYV 640 480 15.0 JeVois DarknetSingle
+##YUYV 1088 480 15.0 YUYV 640 480 15.0 JeVois DarknetSingle
+#
+#####################################################################################################
+#### Detect salient objects in scenes using saliency + darknet deep neural network
+#####################################################################################################
+#
+##YUYV 460 240 15.0 YUYV 320 240 15.0 JeVois DarknetSaliency # not for mac (width not multiple of 16)
+##YUYV 560 240 15.0 YUYV 320 240 15.0 JeVois DarknetSaliency
+#YUYV 880 480 10.0 YUYV 640 480 10.0 JeVois DarknetSaliency
+#
+#####################################################################################################
+#### FIRST robotics object detection example in C++
+#####################################################################################################
+#
+##YUYV 352 194 120.0 YUYV 176 144 120.0 JeVois FirstVision
+##YUYV 176 194 120.0 YUYV 176 144 120.0 JeVois FirstVision
+#YUYV 640 290 60.0 YUYV 320 240 60.0 JeVois FirstVision
+#YUYV 320 290 60.0 YUYV 320 240 60.0 JeVois FirstVision
+#NONE 0 0 0.0 YUYV 320 240 60.0 JeVois FirstVision
+#NONE 0 0 0.0 YUYV 176 144 120.0 JeVois FirstVision
+#
+#####################################################################################################
+#### FIRST robotics object detection example in Python
+#####################################################################################################
+#
+#YUYV 640 252 60.0 YUYV 320 240 60.0 JeVois FirstPython
+##YUYV 320 252 60.0 YUYV 320 240 60.0 JeVois FirstPython
+#NONE 0 0 0.0 YUYV 320 240 60.0 JeVois FirstPython
+#NONE 0 0 0.0 YUYV 176 144 120.0 JeVois FirstPython
+#
+#####################################################################################################
+#### Object detection using SURF and 6D pose estimation
+#####################################################################################################
+#
+#YUYV 320 262 15.0 YUYV 320 240 15.0 JeVois PythonObject6D
+##YUYV 640 502 10.0 YUYV 640 480 10.0 JeVois PythonObject6D
+#NONE 0 0 0.0 YUYV 320 240 15.0 JeVois PythonObject6D
+#
+#####################################################################################################
+#### Edge detection using 4 Canny filters in parallel, with different settings, example python parallel
+#####################################################################################################
+#
+## Disabled by default because Python multiprocessing is very buggy. Note that enabling this mapping may also
+## render your JeVois camera undetectable by OSX hosts.
+#
+##YUYV 1280 240 30.0 YUYV 320 240 30.0 JeVois PythonParallel
+#
+#####################################################################################################
+#### Detect objects in scenes using tensorflow deep neural network
+#####################################################################################################
+#
+##YUYV 560 240 15.0 YUYV 320 240 15.0 JeVois TensorFlowSingle
+#YUYV 464 240 15.0 YUYV 320 240 15.0 JeVois TensorFlowSingle
+##YUYV 880 480 15.0 YUYV 640 480 15.0 JeVois TensorFlowSingle
+##YUYV 1104 480 15.0 YUYV 640 480 15.0 JeVois TensorFlowSingle
+#
+#####################################################################################################
+#### Detect salient objects in scenes using saliency + tensorflow deep neural network
+#####################################################################################################
+#
+##YUYV 448 240 30.0 YUYV 320 240 30.0 JeVois TensorFlowSaliency
+#YUYV 512 240 30.0 YUYV 320 240 30.0 JeVois TensorFlowSaliency
+##YUYV 544 240 30.0 YUYV 320 240 30.0 JeVois TensorFlowSaliency
+#
+#####################################################################################################
+#### Detect objects in scenes using tensorflow deep neural network, easy version
+#####################################################################################################
+#
+#YUYV 320 308 30.0 YUYV 320 240 30.0 JeVois TensorFlowEasy
+##YUYV 640 548 30.0 YUYV 640 480 30.0 JeVois TensorFlowEasy
+#YUYV 1280 1092 7.0 YUYV 1280 1024 7.0 JeVois TensorFlowEasy
+#
+#####################################################################################################
+#### ArUco augmented-reality markers detection and decoding + color blob detection
+#####################################################################################################
+#
+#YUYV 320 266 30.0 YUYV 320 240 30.0 JeVois ArUcoBlob
+#
+#####################################################################################################
+#### Detect and identify objects in scenes using OpenCV DNN detection framework
+#####################################################################################################
+#
+#YUYV 640 502 20.0 YUYV 640 480 20.0 JeVois PyDetectionDNN
+#YUYV 640 498 15.0 YUYV 640 480 15.0 JeVois DetectionDNN
+#
+#####################################################################################################
+#### Simple demo of the ICM-20948 IMU attached to the AR0135 global shutter sensor
+#####################################################################################################
+#
+#YUYV 640 512 40.0 YUYV 640 480 40.0 JeVois DemoIMU
+#
+#####################################################################################################
+#### Object classification using OpenCV DNN in Python
+#####################################################################################################
+#
+#YUYV 320 264 30.0 YUYV 320 240 30.0 JeVois PyClassificationDNN
+#
+#####################################################################################################
+#####################################################################################################
+#####################################################################################################
+#####################################################################################################
+## Modules provided by jevoisextra
+#####################################################################################################
+#####################################################################################################
+#####################################################################################################
+#####################################################################################################
+#
+#YUYV 320 264 15.0 YUYV 320 240 15.0 JeVois FaceDetector
+#
diff --git a/y2019/image_streamer/vision.conf b/y2019/image_streamer/vision.conf
new file mode 100644
index 0000000..df789dd
--- /dev/null
+++ b/y2019/image_streamer/vision.conf
@@ -0,0 +1,5 @@
+[program:vision]
+command=/root/image_streamer --single_camera=true --camera0_exposure=600 --roborio_ip=10.9.71.2
+redirect_stderr=false
+autostart=true
+autorestart=true
diff --git a/y2019/vision.proto b/y2019/vision.proto
new file mode 100644
index 0000000..fb1eeec
--- /dev/null
+++ b/y2019/vision.proto
@@ -0,0 +1,12 @@
+syntax = "proto2";
+
+package y2019;
+
+message VisionControl {
+ optional bool high_video = 1;
+}
+
+message VisionStatus {
+ optional uint32 high_frame_count = 1;
+ optional uint32 low_frame_count = 2;
+}