public
/
image2rtsp
mirror of https://github.com/maladzenkau/image2rtsp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

refactored gstreamer pipeline

This commit is contained in:
Dzmitry Maladzenkau 2025-05-31 17:27:49 +02:00
parent 9d7c6a10c3
commit c812ce9ada
5 changed files with 173 additions and 97 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

74
README.md
View File

@ -2,17 +2,17 @@
This project is a migration from ROS1 to ROS2. The original code was developed by [CircusMonkey](https://github.com/CircusMonkey/ros_rtsp/tree/master). I would like to express my gratitude for his contribution. This project is a migration from ROS1 to ROS2. The original code was developed by [CircusMonkey](https://github.com/CircusMonkey/ros_rtsp/tree/master). I would like to express my gratitude for his contribution.
## image2rtsp ## image2rtsp
This project enables the conversion of a selected ROS2 topic of type `sensor_msgs::msg::Image` into an `RTSP` stream, with an anticipated delay of approximately 0,3-0,4s. It also supports usb camera as a direct source. The generated stream can be utilized for various purposes such as remote control, object detection tasks, monitoring and more. This project enables the conversion of a selected ROS2 topic of type `sensor_msgs::msg::Image` or `sensor_msgs::msg::CompressedImage` into an `RTSP` stream, with an anticipated delay of approximately 0,3-0,4s. It also supports usb camera as a direct source. The generated stream can be utilized for various purposes such as remote control, object detection tasks, monitoring and more.
Currently supported `sensor_msgs::msg::Image` formats: "**rgb8**", "**rgba8**", "**rgb16**", "**rgba16**", "**bgr8**", "**bgra8**", "**bgr16**", "**bgra16**", "**mono8**", "**mono16**", "**yuv422_yuy2**". Currently supported and tested `sensor_msgs::msg::Image` formats: "**rgb8**", "**rgba8**", "**rgb16**", "**rgba16**", "**bgr8**", "**bgra8**", "**bgr16**", "**bgra16**", "**mono8**", "**mono16**", "**yuv422_yuy2**".
**Compressed images** are supported as well.
(if you need some specific unsupported format, create an issue and i will try to add it as soon as possible)
The development is being carried out on Ubuntu 20.04 with ROS2 Foxy, also tested on Ubuntu 22.04 with ROS2 Humble. Supported and tested `sensor_msgs::msg::CompressedImage` formats: "**rgb8; jpeg compressed bgr8**". Other formats may work as well with some color scheme deviations. Please open an issue in this case and attach your **ros2 bag**, so i can fix it. If you need some specific unsupported format, create an issue and i will try to add it as soon as possible, but normally it takes pretty long, so dont hesitate to create a PR.
The development is being carried out on Ubuntu 22.04 with ROS2 Humble. Tested with Intel RealSense d435i.
You are reading now the README for a **default** ROS2 package. If you want to use this package written as a ROS2 component, checkout `ros2_component` branch. You are reading now the README for a **default** ROS2 package. If you want to use this package written as a ROS2 component, checkout `ros2_component` branch.
## Dependencies ## Dependencies
- ROS2 Foxy/Humble - ROS2 Humble
- gstreamer libs: - gstreamer libs:
```bash ```bash
@ -29,36 +29,49 @@ sudo apt-get install libgstreamer-plugins-base1.0-dev libgstreamer-plugins-good1
```bashrc ```bashrc
git clone https://github.com/maladzenkau/image2rtsp.git --single-branch git clone https://github.com/maladzenkau/image2rtsp.git --single-branch
``` ```
- Check the framerate of the topic to be subscribed:
```bashrc
ros2 topic hz /someTopic
```
- Adjust `parameters.yaml` according to your needs: - Adjust `parameters.yaml` according to your needs:
```bashrc ```bashrc
gedit ~/ros2_ws/src/image2rtsp/config/parameters.yaml gedit ~/ros2_ws/src/image2rtsp/config/parameters.yaml
``` ```
# Example ROS2 Image topic stream # Example ROS2 Image topic stream
# If the source is a ros2 topic (default case)
compressed: False
topic: "color/image_raw"
default_pipeline: |
( appsrc name=imagesrc do-timestamp=true min-latency=0
max-latency=0 max-bytes=1000 is-live=true !
videoconvert !
videoscale !
video/x-raw, framerate=30/1, width=640, height=480 !
x264enc tune=zerolatency bitrate=500 key-int-max=30 !
video/x-h264, profile=baseline !
rtph264pay name=pay0 pt=96 )
# If camera serves as a source (switched off by default) # Notice: The framerate setting does not affect the RTSP stream — it entirely depends on the ros2 topic frequency.
camera: False # It is included in the pipeline and code for structural reasons. You can likely remove it from the pipeline without impacting the package's behavior.
source: "v4l2src device=/dev/video0"
topic: "/color/image_raw" # The ROS2 topic to subscribe to. Dont change, if you use a camera
# Parameters for both cases # If camera serves as a source
mountpoint: "/back" # Choose the mountpoint for the rtsp stream. camera: False
# This will be able to be accessed from rtsp://<server_ip>/portAndMountpoint camera_pipeline: |
bitrate: "500" ( v4l2src device=/dev/video0 !
framerate: "30" # Make sure that your framerate corresponds to the frequency of a topic you are subscribing to videoconvert !
caps_1: "video/x-raw,framerate=" videoscale !
capr_2: "/1,width=640,height=480" video/x-raw, framerate=30/1, width=640, height=480 !
# Set the caps to be applied after getting the ROS2 Image and before the x265 encoder. Ignore x264enc tune=zerolatency bitrate=500 key-int-max=30 !
# framerate setting here. video/x-h264, profile=baseline !
port: "8554" rtph264pay name=pay0 pt=96 )
local_only: True # True = rtsp://127.0.0.1:portAndMountpoint (The stream is accessible only from the local machine)
# False = rtsp://0.0.0.0:portAndMountpoint (The stream is accessible from the outside) # Notice: Here the framerate might be set to the camera framerate, otherwise "503 Service Unavailable" error will appear.
# For example, to access the stream running on the machine with IP = 192.168.20.20,
# use rtsp://192.186.20.20:portAndMountpoint # RTSP setup
mountpoint: "/back"
port: "8554"
local_only: True # True = rtsp://127.0.0.1:portAndMountpoint (The stream is accessible only from the local machine)
# False = rtsp://0.0.0.0:portAndMountpoint (The stream is accessible from the outside)
# For example, to access the stream running on the machine with IP = 192.168.20.20,
# use rtsp://192.186.20.20:portAndMountpoint
- Save your configuration and navigate to `ros2_ws` colcon root, source and build the package: - Save your configuration and navigate to `ros2_ws` colcon root, source and build the package:
```bashrc ```bashrc
cd ~/ros2_ws/ cd ~/ros2_ws/
@ -83,5 +96,6 @@ cd ~/ros2_ws/
source install/setup.bash source install/setup.bash
ros2 launch image2rtsp rtsp.launch.py ros2 launch image2rtsp rtsp.launch.py
``` ```
## To Do ## Note
- Add compressed images formats (jpeg, png)
- The YAML configuration allows you to fully customize the pipeline according to your needs. This package does not provide any built-in acceleration. As its stability has not been validated across a wide range of Linux systems using advanced hardware or software techniques, support for such configurations is left to the user. There are no plans to update the package to support GPU/CPU acceleration. Please do not open issues related to software/hardware acceleration if they are directly related to the GStreamer pipeline itself.

51
config/parameters.yaml
View File

@ -1,22 +1,39 @@
/image2rtsp: /image2rtsp:
ros__parameters: ros__parameters:
# If camera serves as a source
camera: False
source: "v4l2src device=/dev/video0"
# If the source is a ros2 topic (default case) # If the source is a ros2 topic (default case)
compressed: True compressed: False
topic: "/image_raw/compressed" topic: "color/image_raw"
default_pipeline: |
( appsrc name=imagesrc do-timestamp=true min-latency=0 max-latency=0 max-bytes=1000 is-live=true !
videoconvert !
videoscale !
video/x-raw, framerate=30/1, width=640, height=480 !
x264enc tune=zerolatency bitrate=500 key-int-max=30 !
video/x-h264, profile=baseline !
rtph264pay name=pay0 pt=96 )
# General setup # Notice: The framerate setting does not affect the RTSP stream — it entirely depends on the ros2 topic frequency.
mountpoint: "/back" # It is included in the pipeline and code for structural reasons. You can likely remove it from the pipeline without impacting the package's behavior.
bitrate: "500"
framerate: "30"
caps_1: "video/x-raw, framerate =" # If camera serves as a source
caps_2: "/1,width=640,height=480" camera: False
port: "8554" camera_pipeline: |
local_only: True # True = rtsp://127.0.0.1:portAndMountpoint (The stream is accessible only from the local machine) ( v4l2src device=/dev/video0 !
# False = rtsp://0.0.0.0:portAndMountpoint (The stream is accessible from the outside) videoconvert !
# For example, to access the stream running on the machine with IP = 192.168.20.20, videoscale !
# use rtsp://192.186.20.20:portAndMountpoint video/x-raw, framerate=30/1, width=640, height=480 !
x264enc tune=zerolatency bitrate=500 key-int-max=30 !
video/x-h264, profile=baseline !
rtph264pay name=pay0 pt=96 )
# Notice: Here the framerate might be set to the camera framerate, otherwise "503 Service Unavailable" error will appear.
# RTSP setup
mountpoint: "/back"
port: "8554"
local_only: True # True = rtsp://127.0.0.1:portAndMountpoint (The stream is accessible only from the local machine)
# False = rtsp://0.0.0.0:portAndMountpoint (The stream is accessible from the outside)
# For example, to access the stream running on the machine with IP = 192.168.20.20,
# use rtsp://192.186.20.20:portAndMountpoint

11
include/image2rtsp.hpp
View File

@ -17,17 +17,13 @@ public:
GstRTSPServer *rtsp_server; GstRTSPServer *rtsp_server;
private: private:
string source;
string topic; string topic;
string mountpoint; string mountpoint;
string bitrate;
string framerate;
string caps_1;
string caps_2;
string port; string port;
string pipeline; string pipeline;
string pipeline_head; string default_pipeline;
string pipeline_tail; string camera_pipeline;
uint framerate;
bool local_only; bool local_only;
bool camera; bool camera;
bool compressed; bool compressed;
@ -37,6 +33,7 @@ private:
void rtsp_server_add_url(const char *url, const char *sPipeline, GstElement **appsrc); void rtsp_server_add_url(const char *url, const char *sPipeline, GstElement **appsrc);
void topic_callback(const sensor_msgs::msg::Image::SharedPtr msg); void topic_callback(const sensor_msgs::msg::Image::SharedPtr msg);
void compressed_topic_callback(const sensor_msgs::msg::CompressedImage::SharedPtr msg); void compressed_topic_callback(const sensor_msgs::msg::CompressedImage::SharedPtr msg);
uint extract_framerate(const std::string& pipeline, uint default_framerate);
GstRTSPServer *rtsp_server_create(const string &port, const bool local_only); GstRTSPServer *rtsp_server_create(const string &port, const bool local_only);
GstCaps *gst_caps_new_from_image(const sensor_msgs::msg::Image::SharedPtr &msg); GstCaps *gst_caps_new_from_image(const sensor_msgs::msg::Image::SharedPtr &msg);
rclcpp::Subscription<sensor_msgs::msg::Image>::SharedPtr subscription_; rclcpp::Subscription<sensor_msgs::msg::Image>::SharedPtr subscription_;

122
src/image2rtsp.cpp
View File

@ -9,56 +9,104 @@ using std::placeholders::_1;
Image2rtsp::Image2rtsp() : Node("image2rtsp"){ Image2rtsp::Image2rtsp() : Node("image2rtsp"){
// Declare and get the parameters // Declare and get the parameters
this->declare_parameter("source", "v4l2src device=/dev/video0"); this->declare_parameter("topic", "/color/image_raw");
this->declare_parameter("topic", "/color/image_raw"); this->declare_parameter("mountpoint", "/back");
this->declare_parameter("mountpoint", "/back"); this->declare_parameter("port", "8554");
this->declare_parameter("bitrate", "500"); this->declare_parameter("local_only", true);
this->declare_parameter("framerate", "30"); this->declare_parameter("camera", false);
this->declare_parameter("caps_1", "video/x-raw, framerate ="); this->declare_parameter("compressed", false);
this->declare_parameter("caps_2", "/1,width=640,height=480");
this->declare_parameter("port", "8554");
this->declare_parameter("local_only", true);
this->declare_parameter("camera", false);
this->declare_parameter("compressed", false);
source = this->get_parameter("source").as_string(); this->declare_parameter("default_pipeline", R"(
topic = this->get_parameter("topic").as_string(); ( appsrc name=imagesrc do-timestamp=true min-latency=0 max-latency=0 max-bytes=1000 is-live=true !
mountpoint = this->get_parameter("mountpoint").as_string(); videoconvert !
bitrate = this->get_parameter("bitrate").as_string(); videoscale !
framerate = this->get_parameter("framerate").as_string(); video/x-raw, framerate=30/1, width=640, height=480 !
caps_1 = this->get_parameter("caps_1").as_string(); x264enc tune=zerolatency bitrate=500 key-int-max=30 !
caps_2 = this->get_parameter("caps_2").as_string(); video/x-h264, profile=baseline !
port = this->get_parameter("port").as_string(); rtph264pay name=pay0 pt=96 )
local_only = this->get_parameter("local_only").as_bool(); )");
camera = this->get_parameter("camera").as_bool();
compressed = this->get_parameter("compressed").as_bool(); this->declare_parameter("camera_pipeline", R"(
( v4l2src device=/dev/video0 !
videoconvert !
videoscale !
video/x-raw, framerate=30/1, width=640, height=480 !
x264enc tune=zerolatency bitrate=500 key-int-max=30 !
video/x-h264, profile=baseline !
rtph264pay name=pay0 pt=96 )
)");
topic = this->get_parameter("topic").as_string();
mountpoint = this->get_parameter("mountpoint").as_string();
port = this->get_parameter("port").as_string();
local_only = this->get_parameter("local_only").as_bool();
camera = this->get_parameter("camera").as_bool();
compressed = this->get_parameter("compressed").as_bool();
default_pipeline = this->get_parameter("default_pipeline").as_string();
camera_pipeline = this->get_parameter("camera_pipeline").as_string();
// Start the subscription // Start the subscription
if (compressed == false){ if (camera == false){
subscription_ = this->create_subscription<sensor_msgs::msg::Image>(topic, 10, std::bind(&Image2rtsp::topic_callback, this, _1)); if (compressed == false){
subscription_ = this->create_subscription<sensor_msgs::msg::Image>(topic, 10, std::bind(&Image2rtsp::topic_callback, this, _1));
RCLCPP_INFO(this->get_logger(), "Subscribing to sensor_msgs::msg::Image");
}
else {
subscription_compressed_ = this->create_subscription<sensor_msgs::msg::CompressedImage>(topic, 10, std::bind(&Image2rtsp::compressed_topic_callback, this, _1));
RCLCPP_INFO(this->get_logger(), "Subscribing to sensor_msgs::msg::CompressedImage");
}
} }
else{ else {
subscription_compressed_ = this->create_subscription<sensor_msgs::msg::CompressedImage>(topic, 10, std::bind(&Image2rtsp::compressed_topic_callback, this, _1)); RCLCPP_INFO(this->get_logger(), "Trying to access camera device");
} }
// Start the RTSP server // Start the RTSP server
video_mainloop_start(); video_mainloop_start();
rtsp_server = rtsp_server_create(port, local_only); rtsp_server = rtsp_server_create(port, local_only);
appsrc = NULL; appsrc = NULL;
// Setup the pipeline
pipeline_tail = "key-int-max=30 ! video/x-h264, profile=baseline ! rtph264pay name=pay0 pt=96 )"; pipeline = camera ? camera_pipeline : default_pipeline;
if (camera == false){ framerate = extract_framerate(pipeline, 30);
pipeline_head = "( appsrc name=imagesrc do-timestamp=true min-latency=0 max-latency=0 max-bytes=1000 is-live=true ! videoconvert ! videoscale ! "; rtsp_server_add_url(mountpoint.c_str(), pipeline.c_str(), camera ? nullptr : (GstElement **)&appsrc);
pipeline = pipeline_head + caps_1 + framerate + caps_2 + " ! x264enc tune=zerolatency bitrate=" + bitrate + pipeline_tail;
rtsp_server_add_url(mountpoint.c_str(), pipeline.c_str(), (GstElement **)&(appsrc));
}
else {
pipeline = "( " + source + " ! videoconvert ! videoscale ! " + caps_1 + framerate + caps_2 + " ! x264enc tune=zerolatency bitrate=" + bitrate + pipeline_tail;
rtsp_server_add_url(mountpoint.c_str(), pipeline.c_str(), NULL);
}
RCLCPP_INFO(this->get_logger(), "Stream available at rtsp://%s:%s%s", gst_rtsp_server_get_address(rtsp_server), port.c_str(), mountpoint.c_str()); RCLCPP_INFO(this->get_logger(), "Stream available at rtsp://%s:%s%s", gst_rtsp_server_get_address(rtsp_server), port.c_str(), mountpoint.c_str());
} }
uint Image2rtsp::extract_framerate(const std::string& pipeline, uint default_framerate = 30) {
std::string search_str = "framerate=";
size_t pos = pipeline.find(search_str);
if (pos == std::string::npos) {
RCLCPP_WARN(this->get_logger(), "Framerate not found in pipeline, using default: %d", default_framerate);
return default_framerate;
}
pos += search_str.length();
size_t end_pos = pipeline.find_first_of("/,", pos);
if (end_pos == std::string::npos) {
RCLCPP_WARN(this->get_logger(), "Invalid framerate format in pipeline, using default: %d", default_framerate);
return default_framerate;
}
std::string framerate_str = pipeline.substr(pos, end_pos - pos);
framerate_str.erase(0, framerate_str.find_first_not_of(" \t"));
framerate_str.erase(framerate_str.find_last_not_of(" \t") + 1);
try {
uint framerate = std::stoi(framerate_str);
if (framerate <= 0) {
RCLCPP_WARN(this->get_logger(), "Invalid framerate value %d, using default: %d", framerate, default_framerate);
return default_framerate;
}
RCLCPP_INFO(this->get_logger(), "Using set framerate %d", framerate);
return framerate;
} catch (const std::exception& e) {
RCLCPP_WARN(this->get_logger(), "Failed to parse framerate '%s', using default: %d", framerate_str.c_str(), default_framerate);
return default_framerate;
}
}
int main(int argc, char *argv[]){ int main(int argc, char *argv[]){
rclcpp::init(argc, argv); rclcpp::init(argc, argv);
auto node = std::make_shared<Image2rtsp>(); auto node = std::make_shared<Image2rtsp>();

10
src/video.cpp
View File

@ -132,7 +132,7 @@ GstCaps *Image2rtsp::gst_caps_new_from_image(const sensor_msgs::msg::Image::Shar
"format", G_TYPE_STRING, format->second.c_str(), "format", G_TYPE_STRING, format->second.c_str(),
"width", G_TYPE_INT, msg->width, "width", G_TYPE_INT, msg->width,
"height", G_TYPE_INT, msg->height, "height", G_TYPE_INT, msg->height,
"framerate", GST_TYPE_FRACTION, stoi(framerate), 1, "framerate", GST_TYPE_FRACTION, framerate, 1,
nullptr); nullptr);
} }
@ -181,9 +181,9 @@ void Image2rtsp::compressed_topic_callback(const sensor_msgs::msg::CompressedIma
// Determine the GStreamer caps // Determine the GStreamer caps
std::string gst_format; std::string gst_format;
switch (img.type()) { switch (img.type()) {
case CV_8UC3: gst_format = "RGB"; break; // RGB images case CV_8UC3: gst_format = "BGR"; break; // BGR images
case CV_8UC4: gst_format = "RGBA"; break; // RGBA images case CV_8UC4: gst_format = "RGBA"; break; // RGBA images
case CV_8UC1: gst_format = "GRAY8"; break; // Grayscale images case CV_8UC1: gst_format = "GRAY8"; break; // Grayscale images
default: default:
RCLCPP_ERROR(this->get_logger(), "Unsupported image type"); RCLCPP_ERROR(this->get_logger(), "Unsupported image type");
return; return;
@ -193,7 +193,7 @@ void Image2rtsp::compressed_topic_callback(const sensor_msgs::msg::CompressedIma
"format", G_TYPE_STRING, gst_format.c_str(), "format", G_TYPE_STRING, gst_format.c_str(),
"width", G_TYPE_INT, img.cols, "width", G_TYPE_INT, img.cols,
"height", G_TYPE_INT, img.rows, "height", G_TYPE_INT, img.rows,
"framerate", GST_TYPE_FRACTION, stoi(framerate), 1, "framerate", GST_TYPE_FRACTION, framerate, 1,
nullptr); nullptr);
// Set caps on appsrc // Set caps on appsrc