Just a little reminder: pip has this very useful option to install a bunch of packages from a single text file mostly called requirements.txt. Anaconda’s command line tool conda doesn’t support this option directly. It does support reading the package names from a file using the –yes and –file option
conda install --yes --file requirements.txt
but that does not automatically install all the dependencies. To do this, we need to iterate over the file and install each package in “single package mode”:
while read requirement; do conda install --yes $requirement; done < requirements.txt
Thanks to Luis Capelo for this snippet which I use to install dependencies in a dockerized instance of Anaconda / Jupyter (more on that in a later post).
This posting will deal with using Docker on the developer desktop. I will not talk about deploying these containers to other stages of the track to production. Maybe this is a topic for a follow-up by me or by someone who is more apt with all things devops.
All this started when I realized, that docker-compose.yml needs an absolute path on the host for its shared volumes. This is OK but when you would like to have multiple development setups for multiple projects. What I wanted was a single config file to rule a complete set of Dockerfile and docker-compose.yml files. And a comandline tool to manage that environment without the need to juggle around with several other tools and numerous options and flags.
An intermediary state consisted of a Makefile with several shell scripts for all the stuff that was hard to do in Makefiles. It worked but was a bunch of files. I wanted something cleaner with more possibilities for the future and fewer helper files.
So here it is: a Python file to rule them all (sorry for the pun …) and build Dockerfile and docker-compose.yml from templates and a config.yml file when booting up the environment. The repository is here: https://github.com/vgoebbels/docker-php7
What you get
- An Apache running PHP7.1 on http://localhost with document root (/var/www/html) as a shared volume in the www subdirectory
- A MySQL database connected to that PHP container
- A PHPMyAdmin listening on http://localhost:8080
- Check out from the Github repo above. Don’t mind the actual path to your environment. This will be determined and inserted into the docker-compose.yml file by the Python script.
- Install the required Python modules with
pip install -r requirements.txt
- Have a look at the templates in the templates subfolder
- Edit the configuration options in config.yml
- Boot the setup using
- Have a look at the running containers with
What doesn’t work yet
Using ./dockshell sshweb and ./dockshell sshsql to log into the running containers. Was not able to enter interactive mode. You will have to use:
docker container exec -it <CONTAINERNAME_HERE> /bin/bash
- ./dockshell clean removes all containers and images. And I mean all of them. This needs to be fixed!
Did you ever stumble across something like this:
Yes, this is strange, isn’t it? The PATH variable is set in the correct order (this is why ‘which’ finds the local Python). Googling about this behavior at first didn’t bring up any solution. But then I came across this now closed question on Stackoverflow.
So once you know what you are looking for Google reveals lots and lots of people having trouble with path hashing. Now, my solution was quite simple:
~ $ type python
python is hashed (/usr/bin/python)
~ $ hash -t python
~ $ hash -d python
~ $ hash -t python
-bash: hash: python: not found
~ $ which python
~ $ python --version
PS: To clear the complete bash path cache just use “hash -r”.
Sometimes you’ll encounter the task to rename files named with snake_case to CamelCase. And sometimes there are a lot of those files. For example when porting a CakePHP 1 project to CakePHP 2 or 3. In CakePHP the upgrade console does a decent job renaming a lot of files for you. But in larger projects having subfolders you’re left with an awful lot of unrenamed scripts. This is where my Python 3 script comes in. It renames any filenames in the current directory given as parameters (thanks to Python 3 argparse you can use wildcards!) from snake_case to CamelCase filename. It preserves the extension if the filename has one. It also has a quiet mode (use -q) to suppress any output at the command line and a preview mode just like GNU make (use -n, supersedes -q), which doesn’t do anything but print out what it would have done.
(Picture by Startup Stock Photos CC 0, http://startupstockphotos.com/)
When using map() you sometimes can be fooled by Pythons lazy evaluation. Many functions returning complex or iterable data don’t do this directly but return a generator object, which when iterated over, yields the result values.
But sometimes you will need the result set at once. For example when map()ing a list one would sometimes coerce Python to return the whole resulting list. This can be done by applying the list() function to the generator like this:
l1=map(lambda x: x+1, l)
<map object at 0x10f4536d8>
l1=map(lambda x: x+1, l)
[2, 3, 4]
In line 5 I have to recreate the map object since print() seems to empty it.
When applying a standard function with map() it’s needed to qualify the module path on call:
l=["Hello ", " World"]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'strip' is not defined
In this case it’s the str module:
Thats all for now. Have fun.