Article
0 comment

Docker automation for PHP Developers using Python

Introduction

This posting will deal with using Docker on the developer desktop. I will not talk about deploying these containers to other stages of the track to production. Maybe this is a topic for a follow-up by me or by someone who is more apt with all things devops.

All this started when I realized, that docker-compose.yml needs an absolute path on the host for its shared volumes. This is OK but when you would like to have multiple development setups for multiple projects. What I wanted was a single config file to rule a complete set of Dockerfile and docker-compose.yml files. And a comandline tool to manage that environment without the need to juggle around with several other tools and numerous options and flags.

An intermediary state consisted of a Makefile with several shell scripts for all the stuff that was hard to do in Makefiles. It worked but was a bunch of files. I wanted something cleaner with more possibilities for the future and fewer helper files.

So here it is: a Python file to rule them all (sorry for the pun …) and build Dockerfile and docker-compose.yml from templates and a config.yml file when booting up the environment. The repository is here: https://github.com/vgoebbels/docker-php7

What you get

  • An Apache running PHP7.1 on http://localhost with document root (/var/www/html) as a shared volume in the www subdirectory
  • A MySQL database connected to that PHP container
  • A PHPMyAdmin listening on http://localhost:8080

Usage

  1. Check out from the Github repo above. Don’t mind the actual path to your environment. This will be determined and inserted into the docker-compose.yml file by the Python script.
  2. Install the required Python modules with
  3. Have a look at the templates in the templates subfolder
  4. Edit the configuration options in config.yml
  5. Boot the setup using
  6. Have a look at the running containers with

What doesn’t work yet

Using ./dockshell sshweb and ./dockshell sshsql to log into the running containers. Was not able to enter interactive mode. You will have to use:

Caveats

  • ./dockshell clean removes all containers and images. And I mean all of them. This needs to be fixed!

 

Article
0 comment

Note to self: bash path hashing /o\

Did you ever stumble across something like this:

Yes, this is strange, isn’t it? The PATH variable is set in the correct order (this is why ‘which’ finds the local Python). Googling about this behavior at first didn’t bring up any solution. But then I came across this now closed question on Stackoverflow.

So once you know what you are looking for Google reveals lots and lots of people having trouble with path hashing. Now, my solution was quite simple:

 

Article
0 comment

Changing filenames to camel case

Sometimes you’ll encounter the task to rename files named with snake_case to CamelCase. And sometimes there are a lot of those files. For example when porting a CakePHP 1 project to CakePHP 2 or 3. In CakePHP the upgrade console does a decent job renaming a lot of files for you. But in larger projects having subfolders you’re left with an awful lot of unrenamed scripts. This is where my Python 3 script comes in. It renames any filenames in the current directory given as parameters (thanks to Python 3 argparse you can use wildcards!) from snake_case to CamelCase filename. It preserves the extension if the filename has one. It also has a quiet mode (use -q) to suppress any output at the command line and a preview mode just like GNU make (use -n, supersedes -q), which doesn’t do anything but print out what it would have done.

(Picture by Startup Stock Photos CC 0, http://startupstockphotos.com/)

Article
0 comment

Practical tips for using map()

When using map() you sometimes can be fooled by Pythons lazy evaluation. Many functions returning complex or iterable data don’t do this directly but return a generator object, which when iterated over, yields the result values.

But sometimes you will need the result set at once. For example when map()ing a list one would sometimes coerce Python to return the whole resulting list. This can be done by applying the list() function to the generator like this:

 

In line 5 I have to recreate the map object since print() seems to empty it.

When applying a standard function with map() it’s needed to qualify the module path on call:

In this case it’s the str module:

 

Thats all for now. Have fun.

Article
0 comment

Note to self: How to count things in Groovy collections

notetoselfThis time I would like to add a short note on how to find things in Groovy collections. Remember: collections is the general term for lists and maps, in other languages sometimes referred to as arrays or dictionaries.

Groovy has a standard method to count all elements of a collection. It is called size():

If you need to know the number of elements in a collection that fit a certain filter, it’s time to switch to count(). Count takes a closure and counts all elements, for which the closure yields true. This can bes as simple as counting all elements larger than 3:

Now what, if the elements of the list are  objects and I want to filter by a specific feature of the objects. No problem:

With maps it’s a bit more tricky. The it object inside the closure is of type LinkedHashMap$Entry, so we have to deal with its key and value attributes:

Hope that helps.  See you next time!

Article
0 comment

Playing around with services in grails console

Suppose you have a grails project and have witten a service doing some database magic to pull together data. Now suppose the very unlikely case that it’s not running that smooth than you thought. To expel the black magic you probably would like to use the grails console to play around with your domain classes and services. Using a service is as simple as importing the domain class and using it:

The service classes however are not that accessible to manipulation. You need to request the service bean instance by name from the application context named ctx:

Remember to use the (lowercase) instance name when calling getBean() just as it would be injected into your controller:

Pulling the strings together you can do more complex tests:

Hope that helps. As always: in case of questions or corrections / additions please leave a comment :)

Article
6 comments

Adding assets in Grails 3

When using modern web development technologies, you often come across frameworks or libraries which use additional resources apart from css stylesheets, images and javascript. One such example is Font Awesome, which needs sone font files, located in the /fonts subdirectory of the unzipped package. In Grails 2 lazy coders would put this directory in the /wep-app folder. In Grails 3 you should (!) use the asset pipeline for these files to and here are two ways that work:

  1. Simply put the files into the grails-app/assets/stylesheets folder. This is not a very elegant way nor is it the intended way to use the asset pipeline.
  2. Put the fonts directory parallel to stylesheets, images and javascript into the grails-app/assets/ folder. For the asset pipeline to know the new directory, specify it in the build.gradle file:

    Last thing to do is to patche the font file paths in the font-awesome.css and/or font-awesome.min.css file. Just remove the “../fonts/” part of the url() path, so they all look like this:

    Thats all.

This post by David Estes put me on the right track, since the official documentation doesn’t mention Grails 3 issues. Thanks David!

Article
0 comment

Separating structure and semantics

There is a great and simple rule, known as “Micha’s Golden Rule”, which goes back to Micha Gorelick (@mynameisfiber). It states:

Do not store data in the keys of a JSON blob.

This means, that instead of writing a JSON dataset holding people and their gaming scores like this:

you should use something like:

First of all, it is good practice, to separate data and its meaning. Second it simplifies software development. And here is why:

One reason is, that in the first form you have no idea, what exactly the number associated with the name means. And when accessing the data you need to know the keys. But the keys are part of the data. So you first have to parse the whole file, separate the keys and iterate over them. In the second case you can iterate over a set of completely identical structured data sets and fetch names and scores.

This rule not only holds true for JSON but for any structured data like XML or yaml. Consider the following modified XML example from the SimpleXML section of the PHP manual:

In PHP you would access the director in this way:

Now if you would like to use the directors name as a key to get the number of oscars he won, it would look like:

This is perfectly valid but stupid XML. And to access the data you need to know the name of the director:

Doesn’t make too much sense, hm? One additional drawback I didn’t mention but that you nevertheless saw: the keys of a data structure language often are subject to several limitaions. In XML element names e.g. there can’t be spaces. So you have to work around that e.g. by camelcasing the name. To get the name back in readable form, you would have to parse it and insert spaces at the correct positions. Which can be impossible with human names, since there are camelcased names like “DeLorean”.

Considering this rule is not always obvious but can save you a lot of nerves. Take care!

Article
0 comment

Starting with Grails and Neo4J

Since Stefan Armbruster is a bit short of time to update the documentation of the Neo4J plugin for Grails I thought I start a short series of postings describing my progress in using Neo4J as a graph database for a Grails based application. I will use the REST API of Neo4J and not the embedded version.
Like always it starts with additions to grails-app/conf/BuildConfig.groovy. First we need to add the Neo4J maven repository to the repositories section:
[pyg l=”groovy”]repositories {
// …
mavenRepo ‘http://m2.neo4j.org/content/repositories/releases/’
} [/pyg]
Then I set my versions:

[pyg]neo4jVersion=”2.0.3″
neo4jRestVersion=”1.9″[/pyg]

 
Now we need to declare the dependencies:
[pyg]dependencies {
//…
compile “org.neo4j:neo4j-community:$neo4jVersion”
compile group:”org.neo4j”, name:”neo4j-graphviz”, version: neo4jVersion
runtime group:”org.neo4j”, name:”neo4j-shell”, version: neo4jVersion
runtime “org.neo4j:neo4j-rest-graphdb:$neo4jRestVersion”
}[/pyg]

En bloc that looks like this (caveat: this code block also installs the angularjs plugin!):

[pyg]grails.project.dependency.resolver = “maven” // or ivy
grails.project.dependency.resolution = {
// inherit Grails’ default dependencies
inherits(“global”) {
}
log “error”
checksums true
legacyResolve false

}[/pyg]

This will get you started with the integration itself. In the next posting I will show, how to access data in the Neo4j database from a controller.