While watching YouTube video, if you hover over the seek bar you can preview video image of that location. You can see images even at 10 min even though only 30 sec of video has buffered yet. So, if the video has not loaded till 10 min, how is the video player able to display image at that location ? Isn’t it interesting ?

This happens with a feature called video scrubbing. We need to process the uploaded video for video scrubbing. When a video is uploaded (say of 10 min), we can take snapshots at regular intervals (of say 3sec). So, for a 10 min video there will be 10*60/3 = 200 snapshots. We can generate these snapshots using ffmpeg.

When you go to HAProxy Site, the small case fonts and hardly any effort on UI make you think do they even maintain this site ? Are HAProxy days done ? But if you make effort to read along, you fumble on this line: Most users report having never ever faced any single crash and claim that HAProxy is the most solid part of their infrastructure. Finding machines with HAProxy processes being up for more than 3 years is not exceptional at all! Phew!! Flexing of highest caliber.

Just as a golf caddie makes the game easier for golfers by carrying their equipments and offering guidance, the Caddy web server aims to make web hosting easier by handling complicated tasks (like SSL/TLS certificates) automatically. With caddy, you don’t need to worry about your certificate expiration. It started back in the days when Let’s Encrypt was just starting and enabling HTTPS on your website was a headache and costly.

Nginx can be used for multiple purposes like web serving, load balancing or reverse proxying. It was designed to solve C10k problem so obviously it can handle large number of connections simultaneously. Besides, it also excels at serving static content. But how does it achieve this ?

Nginx follows an asynchronous event-driven architecture. There is a master process that manages all worker processes. Typically there are as many workers as there are CPU cores. Each worker has an event loop running inside it. When an event comes on socket, OS notifies worker. Worker registers interest in the event and moves ahead. When data is received from client, a read event is triggered which is then processed by worker. Thus, the worker continues with its events without being blocked.

Having worked on Python for 6+ years, learning Rust took me back to C/C++ days in college. Compile the code before running it. With that it also brought back the nostalgic fear of pointers. Rust put all of the fears to rest.

There are two types of memory: Stack and Heap. As the name suggests on stack memory things will be put one above other and popped in reverse order. Since you know where you are putting things and removing from, stack memory is extremely fast.

I have been hearing a lot about Rust lately. How it is fast and achieves near C/C++ performance. So, I was thinking of usecases where I can use it.

I have been using Python (at work) for many years now. Python is a good starting point to embody business ideas as it significantly reduces development time. Libraries are readily available for almost all use cases.

But when your product is built and is handling traffic at scale, you need to go deeper. Python can be a barrier too sometimes. We have migrated one of our high-traffic module from Python (Django) to Java (Spring Webflux) and have seen a 5x impovement in response times. That got me thinking about other options too and eventual interest in Rust.

An Issue was reported that videos were not being uploaded from google drive to S3. We have an asynchronous task that transfers files from google drive to S3 (for further processing).

My first assumption was that the tasks might have been queued. Also there weren’t any sentry alerts, so I ignored the issue. Even after few hours, the issue was not resolved so it seemed something was cooking.

I tried to upload a 2MB file from drive and it got processed instantaneously. So, the pipeline was working fine. I looked at the file that they were trying to upload. It was a whopping 8.3GB file. That raised some doubts. Is google drive blocking such file ? Is some network connection getting reset ? Is S3 limiting file upload ?

We are trying to improve security of our CDN with cloudfront cookies. Cloudfront provides two approaches: one using signed urls and the other using signed cookies. The later suits better for our media usecase where there are lot of fragmented files.

Anyway, I generated a privately signed cookie and sent it in an API response. Then I used this in sending an API request from Postman. And voila it worked. POC was successful.

Now that you have purchased a domain, what will it show ? where will your code reside ? where your images / video files be stored ? how will you run scripts to build your code ?

Let’s go step by step. Say I just want to display an index.html page on my website to host my resume.

1. If you have an AWS account, a simplest way is to create a static website in S3. Create a bucket in S3, create an index.html page in it. Enable static website hosting for your bucket. Create CNAME record in your domain to point to this static website and your resume is up.

Say you want to build a house. Before that you will need to buy a plot (land) where you will build the house.

Similarly, if you want to build something that people can see on internet, you will have to buy some space on internet. This space is domain.

To purchase plot, you need to make registration at registrar office and pay stamp duty fees.

Similar is the process of buying a domain.