Due to the boring school lessons, I decided to participate in GSoC 2020 this year. Since I worked on some Rust projects before, the
FFmpeg + Rust: Code builderGSoC project really caught me. Then I started to do some pre-work. The pre-work consists of several parts and I will talk about the FFI part in this blog post.
FFmpeg project not only provides us ffmpeg, ffplay and ffprobe. They also packed their inner modules for us which can be easily used(relatively). FFmpeg is written in C and that's very convenient for FFI.
First, we need some application code which uses the library. Though I have looked into the libavfilter, I'm not quite familiar with the other ffmpeg interfaces. So I searched for some c code that uses the ffmpeg inner libraries, then I got this. This is a FFmpeg library using tutorial, which is actually pretty good. I choose the first demo to port it into Rust.
This demo uses libavcodec and libavformat. To get proper linting when language porting, function and data structures declaration is needed. So I start to translate the header file into Rust. That's an impossible mission. 5 minutes later I realized that's impossible. Luckily I found the bindgen. It parses the C header file and automatically generates the corresponding Rust expression. The only problem is that C macros are not translated, we need to manually convert them(not quite hard though). With the generated interfaces, I successfully ported the demo into Rust after some fierce fight with unsafe and type inference lol. The only thing make me sad is that the original demo code has some bug which lead to segment fault and took me several hours to figured out that's not my problem... I sent them a pull request to fix it. Souce code is here.
The hardest part is actually building it. My first FFmpeg build is on Windows using msvc(debug a complex codebase with Visual Studio is enjoyable). However, object file generated with clang cannot be linked with msvc lib file. So I migrated to Linux and built FFmpeg libraries with Linux toolchain. The dependencies use in the demo code building process is miscellaneous. I started by finding corresponding dependencies with unresolved external function names and that's also a impossible mission. Then I found that in FFmpeg project the ffbuild/config.mak file generated with the configure script contains compiler parameters for the each library. After turning the compiler parameters to Rust build script, the demo works!
initializing all the containers, codecs and protocols.
opening the input file (bear.mp4) and loading format (container) header
format mov,mp4,m4a,3gp,3g2,mj2, duration 1066667 us, bit_rate 0
finding stream info from format
AVStream->time_base before open coded 1/44100
AVStream->r_frame_rate before open coded 0/0
AVStream->start_time 1024
AVStream->duration 47104
finding the proper decoder (CODEC)
Audio Codec: 2 channels, sample rate 44100
Codec aac ID 86018 bit_rate 74553
AVStream->time_base before open coded 1/30000
AVStream->r_frame_rate before open coded 30000/1001
AVStream->start_time 1001
AVStream->duration 30030
finding the proper decoder (CODEC)
Video Codec: resolution 320 x 180
Codec h264 ID 27 bit_rate 233574
AVPacket->pts 1001
AVPacket->pts 3003
Frame 1 (type=73, size=6355 bytes) pts 1001 key_frame 1 [DTS 0]
AVPacket->pts 2002
Frame 2 (type=66, size=216 bytes) pts 2002 key_frame 0 [DTS 2]
AVPacket->pts 5005
Frame 3 (type=80, size=1028 bytes) pts 3003 key_frame 0 [DTS 1]
AVPacket->pts 4004
Frame 4 (type=66, size=329 bytes) pts 4004 key_frame 0 [DTS 4]
AVPacket->pts 7007
Frame 5 (type=80, size=1223 bytes) pts 5005 key_frame 0 [DTS 3]
AVPacket->pts 6006
Frame 6 (type=66, size=260 bytes) pts 6006 key_frame 0 [DTS 6]
AVPacket->pts 9009
Frame 7 (type=80, size=1160 bytes) pts 7007 key_frame 0 [DTS 5]
releasing all the resources