Just a little reminder: pip has this very useful option to install a bunch of packages from a single text file mostly called requirements.txt. Anaconda’s command line tool conda doesn’t support this option directly. It does support reading the package names from a file using the –yes and –file option
conda install --yes --file requirements.txt
but that does not automatically install all the dependencies. To do this, we need to iterate over the file and install each package in “single package mode”:
while read requirement; do conda install --yes $requirement; done < requirements.txt
Thanks to Luis Capelo for this snippet which I use to install dependencies in a dockerized instance of Anaconda / Jupyter (more on that in a later post).
This posting will deal with using Docker on the developer desktop. I will not talk about deploying these containers to other stages of the track to production. Maybe this is a topic for a follow-up by me or by someone who is more apt with all things devops.
All this started when I realized, that docker-compose.yml needs an absolute path on the host for its shared volumes. This is OK but when you would like to have multiple development setups for multiple projects. What I wanted was a single config file to rule a complete set of Dockerfile and docker-compose.yml files. And a comandline tool to manage that environment without the need to juggle around with several other tools and numerous options and flags.
An intermediary state consisted of a Makefile with several shell scripts for all the stuff that was hard to do in Makefiles. It worked but was a bunch of files. I wanted something cleaner with more possibilities for the future and fewer helper files.
So here it is: a Python file to rule them all (sorry for the pun …) and build Dockerfile and docker-compose.yml from templates and a config.yml file when booting up the environment. The repository is here: https://github.com/vgoebbels/docker-php7
What you get
- An Apache running PHP7.1 on http://localhost with document root (/var/www/html) as a shared volume in the www subdirectory
- A MySQL database connected to that PHP container
- A PHPMyAdmin listening on http://localhost:8080
- Check out from the Github repo above. Don’t mind the actual path to your environment. This will be determined and inserted into the docker-compose.yml file by the Python script.
- Install the required Python modules with
pip install -r requirements.txt
- Have a look at the templates in the templates subfolder
- Edit the configuration options in config.yml
- Boot the setup using
- Have a look at the running containers with
What doesn’t work yet
Using ./dockshell sshweb and ./dockshell sshsql to log into the running containers. Was not able to enter interactive mode. You will have to use:
docker container exec -it <CONTAINERNAME_HERE> /bin/bash
- ./dockshell clean removes all containers and images. And I mean all of them. This needs to be fixed!
Did you ever stumble across something like this:
Yes, this is strange, isn’t it? The PATH variable is set in the correct order (this is why ‘which’ finds the local Python). Googling about this behavior at first didn’t bring up any solution. But then I came across this now closed question on Stackoverflow.
So once you know what you are looking for Google reveals lots and lots of people having trouble with path hashing. Now, my solution was quite simple:
~ $ type python
python is hashed (/usr/bin/python)
~ $ hash -t python
~ $ hash -d python
~ $ hash -t python
-bash: hash: python: not found
~ $ which python
~ $ python --version
Recently someone pointed out, that the perceived number of assholes is rising. This means, that anti-social behavior can be encountered more often. I responded that there seems to be some sort of sociological mechanism leading to anti-social behavior being more accepted than before.
Someone else asked what that mechanism might be, so here is my amateurish point of view.
Social interaction often is accomplished by communication. So examining forms of anti-social or disruptive communicational behavior might help to clarify some points. One form of disruptive communication is the so called “interactive vandalism” (Anthony Giddens, Sociology). Giddens points out, that effective communication or interaction is based on a cooperative behavior of the participants. If one party of an interaction deliberately behaves in a non-cooperative way, this often is encountered as a aggressive attitude by the other participants. But this is a stylistic device, not an explanation.
Another perspective is that of Erving Goffman‘s “The Presentation of Self in Everyday Life“: people behave as if they were acting. And like in a theater there is a stage and a backstage area. In the front region (stage) they act mostly according to common sense rules. In the back region (backstage) they can “give vent to feelings and styles of behavior they keep in check when on stage”. So acting more anti-social might mean transferring behavioral patterns from back to front.
In traditional social settings this would in general have been a completely unacceptable behavior, but why is it not judged that way now? There is a general process at work transferring the private into the public. Reality shows on TV, social media, liberalization of professional situations. Don’t get me wrong: I don’t judge those processes, but if not backed by a so called “good education” things can go wrong unnoticed. If this happens and some sort of “invisible control” doesn’t come into effect, openly visible regulation is a way to prevent unwanted situations.
An area where this currently happens are public conferences. More and more conference hosts issue code of conduct manifests. I got into some serious discussions because I objected that these rules are pretty obvious and stating them so explicitly might be sort of embarrassing for “well-behaved visitors”. Conference organizers assured me that these rules are not so obvious anymore. Maybe in the future we’ll see explicitly stated rules for human interaction more often. While being liberated from unnecessarily rigid forms of social behavior is a good thing, this feels like a cultural loss to me. But than again I just might be getting old.
It’s a common place that you can read nearly any result you want from a statistic. You just have to optimize your mathematical model or cut short the reasoning about the data. There is currently a JAMA publication from the American Medical Society which is cited in many magazines and newspapers (even in German Spiegel) as “22% less risk of colorectal cancer for vegetarians”. No ordinary reader of these reviews will have a look at the original numbers in the publication since it is not freely available (yet another reason for open Publication …). But here they are:
- Vegetarian participants: 40367
Cancer cases: 252
- Nonvegetarian participants: 37292
Cancer cases: 238
This makes for the following relative case numbers:
- Vegetarian: 0,624 / 100 participants
- Nonvegetarian: 0,638 / 100 participants
Or a difference of 0,014 cases per 100 people. This means, if you eat meat your risk to come down with a form of colorectal cancer increases by 0,014 percent. This reads quite different, doesn’t it?
From time to time I come across a sort of dispute or even sometimes war at companies of every size: the central IT department tries to impose a certain hardware or software policy on the coworkers they are entitled to take care of.
Every time this happens there are discussions of BYOD vs. company owned devices. The IT departments claim that they can’t guarantee a certain service level, when they don’t have access to the resources used by the coworkers. The supporters of BYOD argument that using their own chosen hard- and software augments productivity and satisfaction.
I have to confess that I’m a strong campaigner for using my own devices and software at work. But to get some insight into this topic we need to separate different requirements determined by the type of job the employees do:
- Office workers need to get things done. With standard tools. They often are happe to have someone to call if things don’t work like expected or needed.
- Software engineers use their (mostly) laptops to build software. They need some control over the environment they work in. Libraries, databases, IDEs, operating systems. They choose the tools hat get the job done. When things don’t work they are able to fix problems by themselves.
These two roughly separated requirement profiles are opposed by two sorts of enterprise environment:
- Proprietary systems and protocols chosen by the IT departments because they know these systems very well and know how to get support from the provider. Things in this category may contain: Microsoft products (Windows, Exchange, …) or enterprise groupware systems like Novell Groupwise, Lotus Notes etc.
- Open protocols and services offer similar options but with a different type of maintenance.
Both approaches require nearly the same amount of maintenance but of different types. Proprietary systems often offer poor support to clients offside of the mainstream. For example have you ever tried to connect an Apple laptop to a Novell file share? Don’t try. You’ll get mad about getting the right client tools, software incompatibilities and stuff like that.
So there is a natural match for BOYD environments: use standardized protocols and services like NFS, SMB (which both have their origin in proprietary systems …) or mail protocols like SMTP and IMAP.
If your users would like to work without tinkering with software or services: use a centralized management system. This doesn’t naturally contain closed source and proprietary tools. But often it does.
For a company with technologically apt users it’s better to adopt the BOYD way to maximize productivity and user satisfaction. The latter often is no valid point with IT service departments. Then it’s the job of the people whose job it is to provide a suitable working environment for happy colleagues to make the service departments to work they way they are supposed to work.
This seems to be a particular problem in Germany where I often enjoy contact to IT service departments featuring a very self-centric philosophy. The notion of being a service department to help others do their job is not very popular.
Several studies show that companies are seen as more attractive to new employees when they allow BYOD policies.
On the other hand there are security considerations to be taken into account. But I don’t know of any company owned system that prevents willful or even lazy security breaches.
Reading the user forum discussions about proxy configuration for atom can be a bit misleading for users on Windows. Suppose you’re running a recent Windows version. Suppose you’ve installed atom. Suppose you found out that the binary lives in
(version number can be different …)
and suppose you are situated behind a corporate firewall/proxy which prevents you from installing packages and updates.
Looking around you can find postings specifying what to write into your .apmrc config file (which is the config of apm, the atom package manager). Now you look for that file and find it in
Every time you try to write some config to that file, it will be deleted, as it is autogenerated (just as the comment in the file says …).
The file, you are looking for probably is not existant yet. Just create one named
and put in the following content:
https-proxy = http://<PROXY>:<PORT>
http-proxy = http://<PROXY>:<PORT>
strict-ssl = false
Replace and with your values. Save the file, restrt atom and you’re done. Seems hard to distibuish .atom.apmrc and .atom.apm.apmrc some times …
I use a Dell laptop with Ubuntu 15.04 and the VPN NetworkManager seems to be sort of broken. So I guessed I just resort to plain old OpenVPN Config files. And since I’m a very lazy guy I wanted to have some sort of script generating all that stuff for me. First of all you need a prototype template for the config file. There is one that comes with the downloadable zip from Witopa and is called “SampleConfig.txt”. Copy that to “prototype.txt” and change the line
remote [REPLACE WITH SERVER NAME] 1194
Our script will later on replace the “SERVERNAME” with the actual Witopia VPN server names. In the directory where “prototype.txt” lives, create a subdirectory and put the crypto files from the Witopia zip in. These are: ca.crt, ta.key, USERNAME.crt, USERNAME.key. “USERNAME” will be your username (think you guessed that :)
Now create in the prototype directory a file called “createConfigs.sh” with the following content:
rm -f data/*.ovpn
serverlist=`curl -s https://www.witopia.net/?faq-item=openvpn-ssl-gateway-locations | sed -e "s/<[^>]*>//g" | egrep "^vpn"`
for server in $serverlist;
filename=data/`echo $server | cut -d . -f 2 - | sed 's/\([a-z]\)\([a-z]*\)/\U\1\L\2/g'`.ovpn;
echo "Generating $filename";
cat prototype.txt | sed "s/SERVERNAME/$server/g" > $filename
Line 3 cleans up the data directory for the files to come. Line 5 grabs the web page with the VPN server pages from Witopia.net, eliminates the HTML stuff via sed and greps lines starting with “vpn”. Then we loop through the server list and create an OpenVPN config file in data/ for each server, named “City.ovpn”. First we need to build the filename by grabbing the second field of the server name like “vpn.munich.witopia.net”. We cut the city name out and capitalize the first character. This is my personal preference, you can just leave it lower case if you like. Last part is replacing “SERVERNAME” with the actual server naame via sed and putting it in a file with the freshly created name. Thats it.
But if you are as lazy as me you also would like to have a start script which only needs the name of a city to connect you. Here we go:
city=`echo $1 | sed "s/\.ovpn//g" | tr '[:upper:]' '[:lower:]' | sed 's/\([a-z]\)\([a-z]*\)/\U\1\L\2/g'`
if [ -e "$filename" ]
echo "Starting OpenVPN with $city"
sudo openvpn --client --config $filename --ca ca.crt
This script called “start.sh” resides in the data/ directory takes one argument: the name of a city or of a *.ovpn config file. So valid start script calls are:
As you can see line 3 of the script cuts out the city name, casts all characters to lower case and capitalizes the first character. Then we (re-)add the extension “.ovpn” in line 4 and if there is a config file with that name we start the openvpn client. We need to do that as root user so you problably will need to enter your root password when the openvpn is sudoed.
Thats it, folks. Happy networking :)
Welcome to this (independant) international version of my web site. Here I’m going to share some insight into my work. The explanation I like most of what I do is: I have the best job you can imagine. I can play around with new technology and find (often uncommon) solutions for problems of my customers. This is not limited (but includes) software development.
This page is still in design developmment. Nevertheless I would like to start filling the posts. So please bear with me while the design is still … minimal.