Bioinformatics Solutions on AWS

Bioinformatics Solutions on AWS For Exploratory Analysis aws bioinformatics distributed computing hpc python Aug 10, 2020

This is part 1 of a series I have in the works about Bioinformatics Solutions on AWS. Each part of t...

Dask on HPC dask distributed computing hpc parallel computing python Sep 26, 2019

Recently I saw that Dask, a distributed Python library, created some really handy wrappers for

...

Apache Airflow Tutorial – Part 4 DAG Patterns apache airflow distributed computing docker job queues python Mar 21, 2019

Overview

During the previous parts in this series, I introduced Apache Airflow in general, de

...

Apache Airflow Tutorial – Part 3 Start Building apache airflow distributed computing docker job queues python Mar 15, 2019

Overview

If you've read this far you should have a reasonable understanding of the Apache Airflow...

Apache Airflow Tutorial – Part 1 Introduction apache airflow distributed computing docker python Mar 09, 2019

What is Apache Airflow?

Briefly, Apache Airflow is a workflow management system (WMS). It gro

...

Apache Airflow Tutorial – Part 2 Install with Docker apache airflow distributed computing docker job queue python Mar 09, 2019

Install Apache Airflow With Docker Overview

In this part of the series I will cover how to get a

...

Setting up a Local Spark Development Environment using Docker apache spark distributed computing docker python Mar 02, 2019

Every time I want to get started with new tech I figure out how to get a stack up and running that

...

Deploy a Celery Job Queue With Docker – Part 2 Deploy with Docker Swarm on AWS celery distributed computing docker job queue python Feb 13, 2019

Overview

In Part 1 of this series we went over the Celery Architecture, how to separate out t

...

Deploy a Celery Job Queue With Docker – Part 1 Develop celery distributed computing docker job queue python Feb 09, 2019

Overview

In this post I will hopefully show you how to organize a large docker-compose projec

...