Easy Infrastructure Hack: Solving Uneven Load Distribution in Google Cloud for RTMP Ingestion
March 15, 2025
In October 2023, my team and I embarked on an exciting proof of concept that brought together two major platforms: Tokopedia and Instagram. Our mission was to mirror our live shopping events on Tokopedia to Instagram Live, expanding our reach and engaging with a broader audience. This proof of concept was a collaboration with Meta, and despite its tight timeline, we delivered exceptional results.
Building Tokopedia’s Live Streaming Technology
Before diving into the project specifics, it’s important to highlight the foundation we built at Tokopedia. Over the years, we developed our live streaming technologies from scratch, covering every aspect—from transcoding to last-mile delivery. This infrastructure became the backbone of our live shopping experience, enabling us to deliver high-quality streams to our users.
The Challenge: Mirroring to Instagram Live
The collaboration with Meta was initiated with a clear goal: to mirror our live shopping events from Tokopedia directly to Instagram Live. This wasn’t just about streaming the content; it was about maintaining the interactive and immersive experience that Tokopedia offers, albeit with some limitations due to the differences between the platforms.
Given the nature of the project, we had just five days to implement the solution. One of the biggest challenges was obtaining the RTMP ingest URL for Instagram accounts, which isn’t easily accessible to all users. However, with Meta’s assistance, we were able to fast-track this process, which became a crucial key to the project’s success.
Overcoming Challenges with Innovative Solutions
We only had 5 days to deliver this, and we were aware that we couldn’t replicate the fully interactive shopping experience that Tokopedia users enjoyed. However, we didn’t let that stop us from innovating. We implemented a solution to bridge the gap between the two platforms in several areas.
Live Video Mirroring
The core technique we used to mirror the live stream to both platforms was relaying the RTMP ingestion from Tokopedia’s live streaming technologies into Instagram’s RTMP ingest using FFMPEG. This allowed us to broadcast the same live stream simultaneously on both platforms.
The process begins with the Tokopedia broadcaster, who initiates the live stream. This stream, containing both video and audio, is first sent to Tokopedia’s RTMP ingestion server. The RTMP ingest server acts as a receiver for the live stream and prepares the content for further processing. Once the stream is ingested, it is passed on to a live transcoder.
The live transcoder plays an essential role in providing the stream to suit the user’s bandwidth limit . It converts the original video into multiple resolutions and bitrates, ensuring that viewers with various internet speeds can watch the stream without buffering or quality issues. Once the transcoding is complete, the content is delivered to Tokopedia viewers through a content delivery network (CDN), which ensures that the stream reaches the audience with minimal delay, also known as “last mile delivery.”
In order to mirror the live stream onto Instagram, the process employs an RTMP relay. The relay takes the stream from Tokopedia’s RTMP ingest server and duplicates the video and audio content. This duplicated stream is then forwarded to Instagram’s RTMP ingestion server. Essentially, the relay is responsible for copying the live stream and sending it out to a different platform without interrupting the original stream.
Once the stream reaches Instagram’s RTMP ingest server, though I’m not entirely certain about Instagram’s exact process, I assume it works similarly to Tokopedia’s system. The platform likely uses a live transcoder to convert the stream into various formats and bitrates to ensure compatibility across different devices and internet speeds. After transcoding, the stream is delivered to Instagram viewers via their CDN, ensuring smooth playback and a seamless viewing experience.
Shopping Experiences
In the live stream, product tagging presented a unique challenge. Since we couldn’t implement interactive product tagging as we have on Tokopedia, we opted for a workaround by embedding a static layer of product images over the live video stream on Instagram. While this layer wasn’t interactive, it served an essential purpose by providing important context for viewers. It allowed the audience to visually identify the products being discussed by the host during the live stream.
The product image overlay was added directly at the RTMP relay server. Before the stream was restreamed into Instagram’s RTMP ingestion, the relay server embedded the static product image onto the video feed. This way, the stream maintained a seamless appearance, and the product information was clearly displayed for Instagram viewers, even without interactive functionality. Although not as dynamic as Tokopedia’s live shopping features, this approach successfully bridged the gap, delivering key product details in a visually accessible way.
For chat interaction during the live stream on Instagram, we implemented a clever solution by introducing specific keywords, such as “mau” (meaning “I want it”). Viewers could use this keyword in the chat during the live stream to express interest in a product. When someone posted a message containing this keyword, it automatically triggered a mechanism that sent an Instagram Direct Message (DM) to the user. This DM included a link to the product detail page (PDP) on Tokopedia.
This was made possible through Facebook’s open API, which we used to automate the messaging process. By utilizing this integration, we were able to seamlessly redirect Instagram users to Tokopedia’s app, where they could continue their shopping experience and easily purchase the product they showed interest in.
Reflections on the Project
This project was a testament to what can be achieved with a solid technological foundation, and a willingness to innovate under tight constraints. Although we didn’t have the luxury of time to create a fully immersive live shopping experience on Instagram, the solutions we developed enabled us to engage with the Instagram audience and drive traffic back to Tokopedia.
Looking back, this collaboration not only expanded our reach but also pushed us to think creatively about integrating and extending our existing technologies into new platforms. It was a challenging but rewarding experience, and it has expanded our team knowledge to the new possibilities for future live streaming innovations.
Acknowledging the Team Behind the Success
No project, especially one as complex and time-sensitive as this, can be successful without a dedicated and talented team. I want to express my deepest appreciation for the incredible contributors who made this project possible.
Backend Team:
Muhammad Faris Ghanianto, William Phan, Kristopel Martin Lumbantoruan, Dody Suria Wijaya, Ananta Dwi Prasetya Purnayuda, Aditya Putra Budiman, Kevin Marhan Cunis, Edrick Ezra, Roland Lukas Buulolo, Arif Budi Setiawan, Rezky Alamsyah
Frontend (FE):
Luke Reonaldo
Test Engineering (TE):
Haris Yusuf Bakhtiar, Albert Teonando Suhardi
Design:
Rachel Ariella Kurniawan
Product:
Nelson Adiwira Panata, Felicia Liando, Gilbert
Additionally, our work was significantly bolstered by the guidance and support of our tech leaders, Doni Hanafi, Stephanus Tedy and Kevin Christopher. Their leadership and vision were instrumental in steering the project to success.
