Article
2016-03-30
0 comment
by Volker

Changing filenames to camel case

Sometimes you’ll encounter the task to rename files named with snake_case to CamelCase. And sometimes there are a lot of those files. For example when porting a CakePHP 1 project to CakePHP 2 or 3. In CakePHP the upgrade console does a decent job renaming a lot of files for you. But in larger projects having subfolders you’re left with an awful lot of unrenamed scripts. This is where my Python 3 script comes in. It renames any filenames in the current directory given as parameters (thanks to Python 3 argparse you can use wildcards!) from snake_case to CamelCase filename. It preserves the extension if the filename has one. It also has a quiet mode (use -q) to suppress any output at the command line and a preview mode just like GNU make (use -n, supersedes -q), which doesn’t do anything but print out what it would have done.

(Picture by Startup Stock Photos CC 0, http://startupstockphotos.com/)

Article
2016-01-06
0 comment
by Volker

Practical tips for using map()

When using map() you sometimes can be fooled by Pythons lazy evaluation. Many functions returning complex or iterable data don’t do this directly but return a generator object, which when iterated over, yields the result values.

But sometimes you will need the result set at once. For example when map()ing a list one would sometimes coerce Python to return the whole resulting list. This can be done by applying the list() function to the generator like this:

l=[1,2,3]
l1=map(lambda x: x+1, l)
print(l1)
<map object at 0x10f4536d8>
l1=map(lambda x: x+1, l) 
list(l1)
[2, 3, 4]

 

In line 5 I have to recreate the map object since print() seems to empty it.

When applying a standard function with map() it’s needed to qualify the module path on call:

l=["Hello  ", "  World"]
l1=map(strip, l)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'strip' is not defined

In this case it’s the str module:

l1=map(str.strip, l)
list(l1)
['Hello', 'World']

 

Thats all for now. Have fun.

Article
2015-09-25
0 comment
by Volker

Note to self: How to count things in Groovy collections

notetoselfThis time I would like to add a short note on how to find things in Groovy collections. Remember: collections is the general term for lists and maps, in other languages sometimes referred to as arrays or dictionaries.

Groovy has a standard method to count all elements of a collection. It is called size():

l=[1,2,33]
println l.size() // yields 3

If you need to know the number of elements in a collection that fit a certain filter, it’s time to switch to count(). Count takes a closure and counts all elements, for which the closure yields true. This can bes as simple as counting all elements larger than 3:

l=[1,2,33]
println l.count { it>3 } // yields 1

Now what, if the elements of the list are  objects and I want to filter by a specific feature of the objects. No problem:

class obj {
    def i
    def j
    
    def obj(in_i, in_j) {
        i=in_i
        j=in_j
    }
    
    String toString() {
        return "obj($i, $j)"
    }
}

def a=new obj(1,1)
def b=new obj(1,2)
def c=new obj(1,3)
def list=[a, b, c]

println list.count { it.j>1 } // yields 2, i.e. counts b and c

With maps it’s a bit more tricky. The it object inside the closure is of type LinkedHashMap$Entry, so we have to deal with its key and value attributes:

class obj {
    def i
    def j
    
    def obj(in_i, in_j) {
        i=in_i
        j=in_j
    }
    
    String toString() {
        return "obj($i, $j)"
    }
}

def a=new obj(1,1)
def b=new obj(1,2)
def c=new obj(1,3)
def list=[eins: a, zwei: b, drei: c]

println list.count { it.value.j>1 } // again yields 2

Hope that helps.  See you next time!

Article
2015-07-14
0 comment
by Volker

Playing around with services in grails console

Suppose you have a grails project and have witten a service doing some database magic to pull together data. Now suppose the very unlikely case that it’s not running that smooth than you thought. To expel the black magic you probably would like to use the grails console to play around with your domain classes and services. Using a service is as simple as importing the domain class and using it:

import myproject.Domainclass
def instance=Domainclass.get(3)
println instance.id+"\t"+instance.name

The service classes however are not that accessible to manipulation. You need to request the service bean instance by name from the application context named ctx:

def mcs=ctx.getBean("myCoolService")
def allThings=mcs.getAllThings()

Remember to use the (lowercase) instance name when calling getBean() just as it would be injected into your controller:

class GraphController {
    MyCoolService myCoolService
}

Pulling the strings together you can do more complex tests:

import myproject.Domainclass

def mcs=ctx.getBean("myCoolService")
def instance=Domainclass.get(3)
println instance.id+"\t"+instance.name
println "-----------------------------------------"
def allThings=mcs.getAllThings()
allThings.each { n -> println n.id+"\t"+n.thingstype+"\t\t"+n.name }

Hope that helps. As always: in case of questions or corrections / additions please leave a comment :)

Article
2015-07-03
6 comments
by Volker

Adding assets in Grails 3

When using modern web development technologies, you often come across frameworks or libraries which use additional resources apart from css stylesheets, images and javascript. One such example is Font Awesome, which needs sone font files, located in the /fonts subdirectory of the unzipped package. In Grails 2 lazy coders would put this directory in the /wep-app folder. In Grails 3 you should (!) use the asset pipeline for these files to and here are two ways that work:

  1. Simply put the files into the grails-app/assets/stylesheets folder. This is not a very elegant way nor is it the intended way to use the asset pipeline.
  2. Put the fonts directory parallel to stylesheets, images and javascript into the grails-app/assets/ folder. For the asset pipeline to know the new directory, specify it in the build.gradle file: 
    assets {
    minifyJs = true
    minifyCss = true
    includes = ["fonts/*"]
}

    Last thing to do is to patche the font file paths in the font-awesome.css and/or font-awesome.min.css file. Just remove the “../fonts/” part of the url() path, so they all look like this:

    font-family: 'FontAwesome';
src: url('fontawesome-webfont.eot?v=4.3.0');

    Thats all.

This post by David Estes put me on the right track, since the official documentation doesn’t mention Grails 3 issues. Thanks David!

Article
2015-06-19
0 comment
by Volker

Separating structure and semantics

There is a great and simple rule, known as “Micha’s Golden Rule”, which goes back to Micha Gorelick (@mynameisfiber). It states:

Do not store data in the keys of a JSON blob.

This means, that instead of writing a JSON dataset holding people and their gaming scores like this:

{
  "Volker": 100,
  "John": 300
}

you should use something like:

[
  {
    "name": "Volker",
    "score": 100
  },
  {
    "name": "John",
    "score": 300
  }
]

First of all, it is good practice, to separate data and its meaning. Second it simplifies software development. And here is why:

One reason is, that in the first form you have no idea, what exactly the number associated with the name means. And when accessing the data you need to know the keys. But the keys are part of the data. So you first have to parse the whole file, separate the keys and iterate over them. In the second case you can iterate over a set of completely identical structured data sets and fetch names and scores.

This rule not only holds true for JSON but for any structured data like XML or yaml. Consider the following modified XML example from the SimpleXML section of the PHP manual:



 
  PHP: Behind the Parser
  Rasmus Lerdorf
  1

In PHP you would access the director in this way:

movie[0]->director;
?>

Now if you would like to use the directors name as a key to get the number of oscars he won, it would look like:



 
  PHP: Behind the Parser
  1

This is perfectly valid but stupid XML. And to access the data you need to know the name of the director:

movie[0]->RasmusLerdorf;
?>

Doesn’t make too much sense, hm? One additional drawback I didn’t mention but that you nevertheless saw: the keys of a data structure language often are subject to several limitaions. In XML element names e.g. there can’t be spaces. So you have to work around that e.g. by camelcasing the name. To get the name back in readable form, you would have to parse it and insert spaces at the correct positions. Which can be impossible with human names, since there are camelcased names like “DeLorean”.

Considering this rule is not always obvious but can save you a lot of nerves. Take care!

Article
2015-02-20
0 comment
by Volker

Starting with Grails and Neo4J

Since Stefan Armbruster is a bit short of time to update the documentation of the Neo4J plugin for Grails I thought I start a short series of postings describing my progress in using Neo4J as a graph database for a Grails based application. I will use the REST API of Neo4J and not the embedded version.
Like always it starts with additions to grails-app/conf/BuildConfig.groovy. First we need to add the Neo4J maven repository to the repositories section:
[pyg l=”groovy”]repositories {
// …
mavenRepo ‘http://m2.neo4j.org/content/repositories/releases/’
} [/pyg]
Then I set my versions:

[pyg]neo4jVersion=”2.0.3″
neo4jRestVersion=”1.9″[/pyg]

 
Now we need to declare the dependencies:
[pyg]dependencies {
//…
compile “org.neo4j:neo4j-community:$neo4jVersion”
compile group:”org.neo4j”, name:”neo4j-graphviz”, version: neo4jVersion
runtime group:”org.neo4j”, name:”neo4j-shell”, version: neo4jVersion
runtime “org.neo4j:neo4j-rest-graphdb:$neo4jRestVersion”
}[/pyg]

En bloc that looks like this (caveat: this code block also installs the angularjs plugin!):

[pyg]grails.project.dependency.resolver = “maven” // or ivy
grails.project.dependency.resolution = {
// inherit Grails’ default dependencies
inherits(“global”) {
}
log “error”
checksums true
legacyResolve false

repositories {
    inherits true // Whether to inherit repository definitions from plugins

    grailsPlugins()
    grailsHome()
    mavenLocal()
    grailsCentral()
    mavenCentral()
    // uncomment these (or add new ones) to enable
    // remote dependency resolution from public Maven repositories
    //mavenRepo "http://repository.codehaus.org"
    //mavenRepo "http://download.java.net/maven/2/"
    //mavenRepo "http://repository.jboss.com/maven2/"
    mavenRepo 'http://m2.neo4j.org/content/repositories/releases/'
}

neo4jVersion="2.0.3"
neo4jRestVersion="1.9"

dependencies {
    // runtime 'mysql:mysql-connector-java:5.1.29'
    // runtime 'org.postgresql:postgresql:9.3-1101-jdbc41'
    test "org.grails:grails-datastore-test-support:1.0.2-grails-2.4"

    compile "org.neo4j:neo4j-community:$neo4jVersion"
    compile group:"org.neo4j", name:"neo4j-graphviz", version: neo4jVersion
    runtime group:"org.neo4j", name:"neo4j-shell", version: neo4jVersion
    runtime "org.neo4j:neo4j-rest-graphdb:$neo4jRestVersion"
}

plugins {
    // plugins for the build system only
    build ":tomcat:7.0.55"

    // plugins for the compile step
    compile ":scaffolding:2.1.2"
    compile ':cache:1.1.8'
    compile ":asset-pipeline:1.9.9"

    compile ":angularjs:1.0.0"

    // plugins needed at runtime but not for compilation
    runtime ":hibernate4:4.3.6.1" // or ":hibernate:3.6.10.18"
    runtime ":database-migration:1.4.0"
    runtime ":jquery:1.11.1"

}

}[/pyg]

This will get you started with the integration itself. In the next posting I will show, how to access data in the Neo4j database from a controller.