An Issue was reported that videos were not being uploaded from google drive to S3. We have an asynchronous task that transfers files from google drive to S3 (for further processing).

My first assumption was that the tasks might have been queued. Also there weren’t any sentry alerts, so I ignored the issue. Even after few hours, the issue was not resolved so it seemed something was cooking.

I tried to upload a 2MB file from drive and it got processed instantaneously. So, the pipeline was working fine. I looked at the file that they were trying to upload. It was a whopping 8.3GB file. That raised some doubts. Is google drive blocking such file ? Is some network connection getting reset ? Is S3 limiting file upload ?

We are trying to improve security of our CDN with cloudfront cookies. Cloudfront provides two approaches: one using signed urls and the other using signed cookies. The later suits better for our media usecase where there are lot of fragmented files.

Anyway, I generated a privately signed cookie and sent it in an API response. Then I used this in sending an API request from Postman. And voila it worked. POC was successful.

Now that you have purchased a domain, what will it show ? where will your code reside ? where your images / video files be stored ? how will you run scripts to build your code ?

Let’s go step by step. Say I just want to display an index.html page on my website to host my resume.

1. If you have an AWS account, a simplest way is to create a static website in S3. Create a bucket in S3, create an index.html page in it. Enable static website hosting for your bucket. Create CNAME record in your domain to point to this static website and your resume is up.

Say you want to build a house. Before that you will need to buy a plot (land) where you will build the house.

Similarly, if you want to build something that people can see on internet, you will have to buy some space on internet. This space is domain.

To purchase plot, you need to make registration at registrar office and pay stamp duty fees.

Similar is the process of buying a domain.