Running Kubernetes in Offline Mode

Here I'll talk about how to run kubernetes on a flight that doesn't have wifi... or, Red Hat Summit hands on lab that is completely disconnected.  In either case, to set some context, this is useful for me while I'm running on a single host kubernetes configuration for a lab or development where network access is limited or non-existent.

The issue is that K8s tries to pull the pause container whenever it launches a pod.  As such, it tries to connect to gcr.io and make a connection to download the pause image. The gcr.io is the Google Container Registry.  When you are in a disconnected environment this will cause the pod to enter a state of pending until it can pull down the pause container. 

Here's what you can do to bypass that - at least the only thing I know you can do: pull the pause container ahead of time.  It helps if you know you'll be in an environment with limited access ahead of time. 

       
# docker pull gcr.io/google_containers/pause
Trying to pull repository gcr.io/google_containers/pause ...
6c4579af347b: Download complete 
511136ea3c5a: Download complete 
e244e638e26e: Download complete 
Status: Downloaded newer image for gcr.io/google_containers/pause:latest




# docker images
REPOSITORY                       TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
fedora/apache                    latest              1eff270e703a        7 days ago          649.7 MB
gcr.io/google_containers/pause   1.0                 6c4579af347b        11 months ago       239.8 kB
gcr.io/google_containers/pause   go                  6c4579af347b        11 months ago       239.8 kB
gcr.io/google_containers/pause   latest              6c4579af347b        11 months ago       239.8 kB

Extending Storage on an Fedora Atomic Host

I had to spend some time understanding how to use docker-storage-setup on an Atomic host. The tool docker-storage-setup comes by default and makes the configuration of storage on your Atomic host easier. I didn't read any of the provided documentation (although that probably would have helped) other than the script itself.  So, pardon me if this is a duplicate of other info out there.  It was a great way to learn more about it.  The goal here is to add more disk space to an Atomic host.  By default, the cloud image that you download has one device (vda) that is 6GB in size.  When I'm testing many, many docker builds and iterating through the Fedora-Dockerfiles repo, that's just not enough space.  So, I need to know how to expand it.

To provide some context about my environment, I'm using a local KVM environment to hack around in.  The first thing I'll do is go ahead and add a few extra disks to my environment so I can do some testing of docker-storage-setup.  Here is what we will be modifying on our running Atomic VM:

My VM is called: atomic1
New disk 1: vdb (logical name presented to VM)
New disk 2: vdc (logical name presented to VM)
New disk 3: vdd (logical name presented to VM)

As with anything you do regarding storage, make sure you have a backup.

Here is what it looks like on the Atomic VM before I add my disks:

       

# atomic host status
  TIMESTAMP (UTC)         VERSION   ID             OSNAME            REFSPEC                                                
* 2015-06-27 20:22:47     22.50     0eca6e0777     fedora-atomic     fedora-atomic:fedora-atomic/f22/x86_64/docker-host     
  2015-05-21 19:01:46     22.17     06a63ecfcf     fedora-atomic     fedora-atomic:fedora-atomic/f22/x86_64/docker-host     


# fdisk -l | grep vd
Disk /dev/vda: 6 GiB, 6442450944 bytes, 12582912 sectors
/dev/vda1  *      2048   616447   614400  300M 83 Linux
/dev/vda2       616448 12582911 11966464  5.7G 8e Linux LVM

How to Contribute to the "Container Best Practices Guide"

Hey there.  We are starting a new best practices guide for containers!  We'll cover tips and tricks for running containers on Fedora (rkt or Docker), CentOS, Red Hat Enterprise Linux and Atomic.  Some of the topics will cover items from how to build single app containers running on a single host to building containers with the intention of orchestrating them across multiple hosts with a higher level tool like OpenShift and or Kubernetes.

Right now we are just getting started with this consolidation of container knowledge effort.  Please feel free to have a look at the Github repo and contribute by submitting a pull request.  The guide will be written in asciidoc so it's going to be very easy to contribute to.  There are three ways to render the asciidoc files into PDF or HTML format:

• Install the appropriate packages (git asciidoc dockbook-xsl fop make) on your Fedora host
• Build your own container-best-practices (click the link to get the Dockerfile) image and do the processing inside the container
• Pull the trusted image from the Fedora account on the Docker registry by issuing a "docker pull fedora/container-best-practices"

Syntax highlighting for asciidoc

Cool tip to track here.

http://www.methods.co.nz/asciidoc/userguide.html#_vim_syntax_highlighter

To enable syntax highlighing:

• Put a Vim autocmd in your Vim configuration file (see the example vimrc file).
• or execute the Vim command :set syntax=asciidoc.
• or add the following line to the end of you AsciiDoc source files:

  // vim: set syntax=asciidoc:

Flannel and Docker on Fedora - Getting Started

Lets set up 3 Fedora servers for the purposes of testing flannel on Fedora. These can be bare metal, VMs (on KVM, VMware, RHEV, etc...). Why do we want to test this? This is to demonstrate setting up the flannel overlay network and confirming connectivity. Specifically, I want to test container connectivity across hosts.  I'd like to make sure that container A on host A can talk to container B on host B. I received quite a bit of guidance from Jeremy Eder of breakage.org - Thanks for the tips!

Our 3 Flannel hosts:

fed-master 192.168.121.105
fed-minion1 192.168.121.166
fed-minion2 192.168.121.108

A few setup notes: I haven't looked at this on GCE or AWS. It helps to add the hosts to /etc/hosts, or have some other DNS solution. In my case, I set up these VM's in Vagrant on my laptop and modified /etc/hosts.

Software used on these Fedora hosts.

       
# rpm -qa | egrep "etc|docker|flannel"
flannel-0.2.0-1.fc21.x86_64
docker-io-1.4.0-1.fc21.x86_64
etcd-0.4.6-6.fc21.x86_64

On fed-master:

Look at networking before flannel configuration.

       
# ip a

Start etcd on fed-master.

       
# systemctl start etcd; systemctl status etcd

RHEL Atomic Documentation - Get Started

Lots of good stuff to get everyone started.  Lots of options to get started.

Getting Started with Atomic:

https://access.redhat.com/articles/881893

Atomic on RHEV:

https://access.redhat.com/articles/rhel-atomic-install-rhev

Atomic on AWS:

https://access.redhat.com/articles/rhel-atomic-install-aws

Out of Space... Add a disk!

I have a lenovo Thinkpad T540p. This laptop has a Samsung 256GB SSD drive.  This drive ran out of space long ago.  The good thing is, you can add a disk to the T540p.  I went to lenovo's site and ordered:

• ThinkPad 9.5mm SATA Hard Drive Bay Adapter IV
• ThinkPad 500GB 7200rpm 7mm SATA3 Hard Drive 

After installing the drive, I had to figure out how to carve up the new space.  There are probably many ways to do this, but I knew I wanted to:

• Add more space to the /dev/fedora/root LV.
• Keep the new, slower disk in a separate VG.  I didn't want any LV's to span these two disks.

Simple git rebase example

Situation:

So I forked the Google Kubernetes project.  Then I created a fedora_gs_guide branch.  I made some changes to the getting started guide and then I submitted a pull request.  I asked someone to review it and they had a couple of changes.  So, I make the changes, and commit.  Now when I look at the PR, I see multiple commits.  I think it's best practice to squash all those commits into one if possible.  So, how do you do that.  Well, here's how I did it.  I'm sure there are other ways to do this, probably a lot more efficiently (comments welcome).  But, it worked.  These are my notes from the process.

I needed to make sure my master and fedora_gs_guide branch were clean and rebased to upstream master.

Make sure I'm on my local master.

$ git checkout master

Getting Started with Kubernetes / Docker on Fedora

EDIT 9/16/2014 ***********************

I have taken these instructions and put them on the kubernetes github repo:

Kubernetes Gitub

End EDIT ******************


These are my notes on how to get started evaluating a Fedora / Docker / kubernetes environment.  I'm going to start with two hosts.  Both will run Fedora rawhide.  The goal is to stand up both hosts with kubernetes / Docker and use kubernetes to orchestrate the deployment of a couple of simple applications.  Derek Carr has already put together a great tutorial on getting a kubernetes environment up using vagrant.  However, that process is quite automated and I need to set it all up from scratch.

Install Fedora rawhide using the instructions from here.  I just downloaded the boot.iso file and used KVM to deploy the Fedora rawhide hosts.  My hosts names are: fed{1,2}.

The kubernetes package provides four services: apiserver, controller, kubelet, proxy.  These services are managed by systemd unit files. We will break the services up between the hosts.  The first host, fed1, will be the kubernetes master.  This host will run the apiserver and controller.  The remaining host, fed2 will be minions and run kubelet, proxy and docker.

This is all changing rapidly, so if you walk through this and see any errors or something that needs to be updated, please let me know via comments below.

So let's get started.

Getting Started with go

I have been following the progress of Jason and his 180 day coding challenge.  I'm going to try something similar except that I'm going to work on go.  The only problem is, that I can't start for 10 days because of some PTO that I have to take starting tomorrow.  Having said that, I'm throwing down the gauntlet now and when I get back, I'll post everyday on my progress.  A few rules, per Jason's post above - yes, I did steal these directly from him with one change on the first rule:

1. Every business day for a minimum of 30 minutes, I must write code or learn about the tool-chain used in the development process. Documentation about the code does not count.
2. The resulting code must be useful, or it should be code that points towards something that eventually will be. No tweaking indentation, no code re-formatting, and if at all possible no re-factoring. (All these things are permitted, but not as the exclusive work of the day.) Tutorials and working through code examples as a means to learn are allowed.
3. All code must be written before midnight, and after 6AM.
4. The code must be Open Source and posted on Github.

My current status:

I have signed up for the Pluralsight on-line go class here. So far, I have made it to the "Variables, Types and Pointers" section.  We just haven't written much code yet.  So I won't count that.

In addition, I have downloaded and installed IntelliJ IDEA 13.1.3 IDE, community edition.  The on-line class they use this so I figured I'd give it a try.  It was relatively easy to set up, I should write a quick post on what I did for reference.  I'm also going to evaluate the vim plug-ins out there for writing go more efficiently.


So, When I get back I plan on kicking this off full steam ahead.