Introduction to Docker

https://youtu.be/EYNwNlOrpr0?si=huUOpwbUc8oxHjWH

• Delivers software in packages called containers.
• Containers are isolated from one another. (Good for avoiding conflicts, e.g., two different databases running in the same host machine.)
• Containers run in a host machine.
• Containers bundle their own software (even OS), libraries and configuration files.
• Containers communicate with each other using well-defined channels.

Why should we care about docker?

• Reproducibility. (Easy to share and rebuild services.)
• Local experiments.
• Integration tests (CI/CD).
• Running processes on the cloud (AWS Batch, Kubernetes jobs).
• Spark.
• Serverless (AWS Lambda, Google Cloud Functions).

<aside> <img src="/icons/thought-alert_red.svg" alt="/icons/thought-alert_red.svg" width="40px" />

Unfortunately, the course’s introduction to Docker is too simplistic and can be confusing. Docker is a complex tool. If you are a beginner, I’d recommend following a third-party tutorial as a prerequisite for the course.

Some interesting tutorials are:

• https://docker-curriculum.com/
• https://www.codecademy.com/learn/ext-courses/working-with-containers-introduction-to-docker
• https://labs.play-with-docker.com/ </aside>

Ingesting NY Taxi Data to Postgres

https://youtu.be/2JM-ziJt0WI?si=-zTxUwFba6PFn5iz

Using Docker to run a local PostgreSQL database:

docker container run -it \\
-e POSTGRES_USER=root \\
-e POSTGRES_PASSWORD=root \\
-e POSTGRES_DB=ny_taxi \\
-v "./ny_taxi_postgres_data:/var/lib/postgresql/data:rw" \\
-p 5432:5432 \\
--name postgres \\
postgres:latest

<aside> <img src="/icons/warning_yellow.svg" alt="/icons/warning_yellow.svg" width="40px" />

Notice the used image is postgres:latest, which as of today (Jan 2025) is Postgres 17. In the course, they use postgres:13 pointing to version Postgres 13.

The latest suffix is not required to specify the latest image version.

</aside>