Article
0 comment

Knowledge in and out

wp-1456109997332.jpgIn my last post I mused about employer policies concerning training and learning of employees. I would like to use this as an introduction to a series of postings related to knowledge management and innovation in firms. Much has been written and discussed about the creation and finding of knowledge. From a practical point of view and in philosophy, which discusses these topics as epistemology. In this first installment I would like to give an overview of what’s ahead and how I would like to structure my texts.

My first approach was to put all of this in two or three postings. Since this is way too much text to be read comfortably I will divide the topics in a more or less natural way like this:

  1. The idea of human capital in contrast to human resources
  2. Philosophy and theory of knowledge, from information via knowledge to innovation
  3. Flow of knowledge in the firm and exchange with external resources
  4. Knowledge management from a social network point of view
  5. Challenges for human resources departments concerning knowledge management
  6. Impediments for knowledge creation and usage (e.g. Not-Invented-Here-Syndrome, NIH)
  7. The state of open innovation
  8. Simulation of knowledge flow and NIH using cellular automata

What do you think? Interested? Is something missing? Should I reorder the topics? Let me know!

(Picture by Tommy Ellis CC BY-ND 2.0, https://www.flickr.com/photos/tommyellis/6326877778/)

Article
1 comment

Degradation on the job

I remember a conversation with my father-in-law. We were discussing the transition of my freelancing work into an employment. We spoke about the salary and benefits in the contract. He asked, if I had included a surcharge for being on a permanent task. I asked him to explain. And here is the bottom line.

Photo from Death to Stock photos http://deathtothestockphoto.com/

We (in this industry) are knowledge workers. This is a much overly used phrase. It boils down to that our value as an employee or freelancer depends on our knowledge, be it explicit or tacit. If we are hired for a certain position, we need to keep at it. There are several possibilities how we can do that. Going to conferences, taking a training, meet colleagues etc. This needs time. And sometimes money. All this expenditure has to be accounted for. By you or by your employer. He may support you with money or free time or he may not (which is common in Germany). If not the salary has to compensate for your personal investment. So why has he to do that?

Because his order to work on a certain task or field for a (supposedly) very long time distracts you from acquiring additional knowledge. If you would still be freelancing, hopping from gig to gig you would to some extent learn new things. And you would invest some of your time and / or money in getting new things to know. Since you are your own boss.

If you work on a defined focussed task for a longer time, you will be left behind on your fields of expertise. This once was the reason you were hired. After some years, your worth as a coworker will degrade. And this is the reason good employers care for your knowledge.

Article
1 comment

How to merge two WordPress blogs

Introduction and Overview

For some of you it might be pretty obvious how to do that. But for some of my readers this might be a new topic. So here is a short overview of how to do it:

  1. Test if it works:
    1. Prequisite: you’ll need a locally running web server with PHP support. If you don’t have one, you could try to test the merge on a second web server, but alas that’s out of the scope of this posting.
    2. Download the destination blog and install it locally.
    3. Install the WordPress Command Line Tool.
    4. Download all articles from the source blog as XML file.
    5. Install the WordPress Import plugin.
    6. Try to import the XML file.
  2. If all works well, repeat that (without step 3 since the URLs are ok for production) in the production instance of the destination blog.

Local test

Download destination blog

First we need a download of the blog, where all postings should be merged together. This means:

  • Zip or tar the whole WordPress directory together, get it down to your machine.
  • Install it as a virtual host on your local web server.
  • Get a full export of your WordPress database.

My WordPress directory was called “www” and resided in /opt/sites/de/technologyscout (that’s how all my web server document roots  are organized /opt/sites/TLD/DOMAIN/SUBDOMAIN). So I downloaded it to my laptop, moved it to /opt/sites (yes, I maintain that also locally) and renamed it to “techscout”.

If you’re like me, it’s a good idea, to look up the database credentials in WORDPRESSROOT/wp-config.php:

define('DB_NAME', 'xxxxx');

define('DB_USER', 'yyyyy');

define('DB_PASSWORD', 'zzzzz');

I also like to have exactly the same credetials locally, so at the MySQL command line I do:

grant all on xxxxx.* to yyyyy@localhost identified by 'zzzzz';

flush privileges;

Then I import the downloaded WordPress database with:

mysql -uyyyyy -pzzzzz xxxxx < database.sql

 

Install WordPress Command Line Tool

Since WordPress has a habit of saving absolute URLs in the database, which is btw. a very bad and stupid habit, we need to patch those URLs to work locally. This is done with the indispensable wp-cli tool:

wp search-replace 'http://www.technologyscout.de' 'http://techscout'

This will replace every occurrence of http://www.technologyscout.de with http://techscout. Now please try to log in with your credentials. Everything should work locally now. Please have a look at the URL in your browser after login. If you see the productive URL (for me http://www.technologyscout.de) this means the search and replace didn’t work as expected.

Download XML archive with all postings

Please go to Tools -> Export in the admin and select “Posts”. You’ll see additional options. I selected:

  • Categories: All
  • Authors: All
  • Start date, End date: Nothing, just go with –Select–
  • Status: All (You can only import published posts, if you like)

Now click on “Download Export File” and save the file locally.

Install WordPress Import plugin

Now please go to Tools -> Import inside the admin panel. You will see a list of options similar to this one:

Importoptions

Please select the last entry called “WordPress” (who could guess that, right?). If you didn’t have the WordPress import plugin installed WordPress will ask you if you would like to do so right now. We certainly want to. After installing the plugin, we are ready to …

Import the WordPress XML file

Proceed by selecting the input file we just downloaded. Everything should work smoothly. The imported postings should be sorted according to their original date into the database.

Do it in production

It’s a good idea to get a database backup of your production destination database before importing the blog posts. Then go to Tools -> Import, install the import plugin if needed, and import the XML file we already used locally.

That’s all. Have fun!

Article
0 comment

Blogumzug – Blog move

Deutsch

Hallo, vielleicht wundern Sie sich, daß Sie auf dieser Seite landen, obwohl Sie die deutsche Version sehen wollten. Ich habe festgestellt, daß es auf Dauer eher unsinnig ist, zwei Blogs mit ähnlichen Inhalten zu verwalten und habe daher die beiden Blogs zusammen gelegt. Die Inhalte von technologyscout.de finden Sie jetzt auch hier auf der Seite.

English

Hi, you might wonder why you got here instead of seeing the german version you expected. I realized, that it is a bit of a hassle to support two blogs with very similar content, so I merged them. So now you can find the content from technologyscout.de in here.

Article
0 comment

Junk in, junk out

When I was at school – yes, this was some time ago – I had a colleague who was at constant war with his frensh language teacher and the teacher with him. So once when writing an exam, in the end my colleague returned a piece of paper to the teachers desk, containing a longer monologue sayingwhy the student absolutely didn’t feel like writing an exam. The whole text was written in a large spiral on the paper.

When the exams were returned, my colleague got his paper back, signed by a note of “6”, which in Germany is the worst note you can get. The explanation of why the teacher voted for a 6 was written around the figure “6” … in a spiral.


One story I read on the internets said, that a student of english literature got back his very very bad exam with a remark from the teaher:

I return this otherwise good writing paper back to you, because someone wrote gibberish all over it and put your name on top.

I like this sense of humour!

Article
0 comment

Starting with Grails and Neo4J

Since Stefan Armbruster is a bit short of time to update the documentation of the Neo4J plugin for Grails I thought I start a short series of postings describing my progress in using Neo4J as a graph database for a Grails based application. I will use the REST API of Neo4J and not the embedded version.
Like always it starts with additions to grails-app/conf/BuildConfig.groovy. First we need to add the Neo4J maven repository to the repositories section:
[pyg l=”groovy”]repositories {
// …
mavenRepo ‘http://m2.neo4j.org/content/repositories/releases/’
} [/pyg]
Then I set my versions:

[pyg]neo4jVersion=”2.0.3″
neo4jRestVersion=”1.9″[/pyg]

 
Now we need to declare the dependencies:
[pyg]dependencies {
//…
compile “org.neo4j:neo4j-community:$neo4jVersion”
compile group:”org.neo4j”, name:”neo4j-graphviz”, version: neo4jVersion
runtime group:”org.neo4j”, name:”neo4j-shell”, version: neo4jVersion
runtime “org.neo4j:neo4j-rest-graphdb:$neo4jRestVersion”
}[/pyg]

En bloc that looks like this (caveat: this code block also installs the angularjs plugin!):

[pyg]grails.project.dependency.resolver = “maven” // or ivy
grails.project.dependency.resolution = {
// inherit Grails’ default dependencies
inherits(“global”) {
}
log “error”
checksums true
legacyResolve false

repositories {
    inherits true // Whether to inherit repository definitions from plugins

    grailsPlugins()
    grailsHome()
    mavenLocal()
    grailsCentral()
    mavenCentral()
    // uncomment these (or add new ones) to enable
    // remote dependency resolution from public Maven repositories
    //mavenRepo "http://repository.codehaus.org"
    //mavenRepo "http://download.java.net/maven/2/"
    //mavenRepo "http://repository.jboss.com/maven2/"
    mavenRepo 'http://m2.neo4j.org/content/repositories/releases/'
}

neo4jVersion="2.0.3"
neo4jRestVersion="1.9"

dependencies {
    // runtime 'mysql:mysql-connector-java:5.1.29'
    // runtime 'org.postgresql:postgresql:9.3-1101-jdbc41'
    test "org.grails:grails-datastore-test-support:1.0.2-grails-2.4"

    compile "org.neo4j:neo4j-community:$neo4jVersion"
    compile group:"org.neo4j", name:"neo4j-graphviz", version: neo4jVersion
    runtime group:"org.neo4j", name:"neo4j-shell", version: neo4jVersion
    runtime "org.neo4j:neo4j-rest-graphdb:$neo4jRestVersion"
}

plugins {
    // plugins for the build system only
    build ":tomcat:7.0.55"

    // plugins for the compile step
    compile ":scaffolding:2.1.2"
    compile ':cache:1.1.8'
    compile ":asset-pipeline:1.9.9"

    compile ":angularjs:1.0.0"

    // plugins needed at runtime but not for compilation
    runtime ":hibernate4:4.3.6.1" // or ":hibernate:3.6.10.18"
    runtime ":database-migration:1.4.0"
    runtime ":jquery:1.11.1"

}

}[/pyg]

This will get you started with the integration itself. In the next posting I will show, how to access data in the Neo4j database from a controller.

Article
0 comment

From JSON to CSV to SQL

The task

I recently had the problem, that I had to insert JSON data sets into a PostgreSQL database. As rows, not as JSON field type.

The JSON represents a network of nodes and links and looked like this:

{
  "links": [
    {
      "source": "22",
      "target": "17",
      "value": 1
    },
    {
      "source": "18",
      "target": "14",
      "value": 1
    },
...
  ],
  "nodes": [
    {
      "name": "Resource 32",
      "image": "/mydemo/images/icon1.svg",
      "group": 1,
      "image_width": 45,
      "image_height": 45
    },
    {
      "name": "Resource 17",
      "image": "/mydemo/images/icon2.svg",
      "group": 1,
      "image_width": 45,
      "image_height": 45
    },
...
  ]
}

What I needed:

  1. 1. split JSON into nodes and links separately
  2. 2. convert to CSV
  3. 3. add an ID field as first column and primary key
  4. 4. number data rows with ID from 1 to n
  5. 5. delete the ugly path “/mydemo/images/”, leaving only the filename
  6. 6. create table in PostgreSQL DB and insert data

Create CSV from JSON

For this I used the brilliant Python based cvskit package. This is done in 2 steps:

The first step completes tasks 1 to 5:

underbrace{tt in2csv -k nodes myfile.json}_{mbox{Term 1}}nolinebreak[4]midnolinebreak[4]underbrace{tt nl -v0 -s,}_{mbox{Term 2}}nolinebreak[4]midnolinebreak[4]underbrace{tt perl -pequad's/hat{quad}{backslash}s+//'}_{mbox{Term 3}}nolinebreak[4]midnolinebreak[4]underbrace{tt sedquad"s/hat{quad}0,/id,/g;s#/visdemo/images/##g"}_{mbox{Term 4}}nolinebreak[4]>nolinebreak[4]mbox{tt nodes.csv}

Term 1: Export subarray with key nodes.
Term 2: Number lines starting with zero (uses unix tool nl). Delimiter is colon.
Term 3: Strip leading spaces.
Term 4: Replace “0,” in first line with col name, strip image path.

The same for the links section of the JSON file:

underbrace{tt in2csv -k links graph3345.json}_{mbox{Term 1}}nolinebreak[4]midnolinebreak[4]underbrace{tt nl -v0 -s,}_{mbox{Term 2}}nolinebreak[4]midnolinebreak[4]underbrace{tt perl -pequad's/hat{quad}{backslash}s+//'}_{mbox{Term 3}}nolinebreak[4]midnolinebreak[4]underbrace{tt sedquad"s/hat{quad}0,/id,/g"}_{mbox{Term 4}}nolinebreak[4]>nolinebreak[4]mbox{tt links.csv}

Term 1: Export subarray with key links.
Term 2: Number lines starting with zero (uses unix tool nl). Delimiter is colon.
Term 3: Strip leading spaces.
Term 4: Replace “0,” in first line with col name.

Insert CSV into database

Now we need another tool from the csvkit tool chain, which reads CSV and builds an SQL CREATE TABLE statement and inserts the data into the created table.

csvsql --db postgresql://user:passwd@localhost/databasename --table node --insert nodes.csv
csvsql --db postgresql://user:passwd@localhost/databasename --table link --insert links.csv

 

The parameters are:

  • ● -db <DBURL> The database URL to connect to, in SQLAlchemy format
  • ● –table <TABLENAME> What name should the table get? if not supplied, the CSV filename without extension will be used.
  • ● –insert not only create the CREATE TABLE DDL statement but also insert the data sets into the freshly created table.

This tool has (in my opinion) three small drawbacks:

  1. 1. It doesn’t allow you to see the insert statements. The CREATE TABLE statement will be printed to stdout, if you don’t provide a db URL. But there is no way to see the insert statements.
  2. 2. It can’t handle already existing tables. There is no –add-drop option to issue a DROP TABLE statement in advance to create the new one.
  3. 3. It’s not able to autocreate an ID column. This complicates the first step of creating a usable CSV file to some extend, as you can see above.

So the last action is to create the primary key on the id column in both tables:

alter table node add primary key (id);
alter table link add primary key (id);

 

Even with these little drawbacks the csvkit tool chain has numerous applications. Give it a spin!

Article
0 comment

Decrease in Internet Explorer usage leads to lowering of murder rate

… well, certainly not. But a brillant article (because it’s as short as possible, consisting simply of an illustration) of Chris Blattman shows, how data and information can be used in a most misleading way. Correlating two sets of data leading to a seemingly interdependence. His example is the decrease of the rate of murder compared to the decrease of Internet Explorer usage. Speak it out loud and you’ll think: thats the dumbest shit I’ve ever heard. But look at his correlation graph. Doesn’t it look very persuading?
That is what I mostly dislike in the current trend to info graphics: ever graphically capable designer, blogger, journalist or whatever is able to create an info graphic not of “reality” (because there is no such thing as absolte truth, but thats a different post about epistemology…)  but of his preconception of “how things are”.
I’m not criticizing the use of info graphics at all, but we should be aware that they not only contain pure data but also the authors opinion of what that means. In the case of Chris Blattmans correlation there are just two sigmoidal curves. Layering them this way expresses an explicit wish to find a common cause for both. But sigmoidal curves just happen to be found nearly everywhere where things in- or decrease.
Having said that, I only can agree that the constant usage of Microsoft software can lead to a very special mind setting :)

[Update on 6. Jan. 2014] Here is another brilliant example of correlation vs. causation: “On correlation, causation, and the “real” cause of autism“.

Article
0 comment

Why you should not deploy web applications with unix packaging tools

Packaging and deploying web applications, e.g. PHP apps, can be a very tough topic. One way to deal with automatic deployment and dependencies is to use the packaging tools your unix server supplies, e.g. debian *.deb packages. This approach has indisputable benefits:

  • it manages dependencies
  • it’s possible to manage the deployment process on several servers automatically
  • there is a complete existing infrastructure for this task

But (you knew that would come, didn’t you?): packaging systems like the one used on debian are targeted to control that the package is installed in exactly one version (certainly not always, we’ll come to that later on) in one defined place. Igf we use multiple versions of the same software, it’s version number will define where they will be installed. You get a version tree like:

/usr/local/lib/…
…/applib-1.1/
…/applib-1.2/
etc.

Most system packaging tools also allow you to specify a prefix where to install the package. But altogether its a special tool for a special task.
Using it for web application deployment looks like the “if all you know is a hammer, every task looks like a nail” paradigm. You can accomplish the tasks needed but you have to modify the standard processes very much.
For example I would like to be able to install exactly the same software twice on exactly the same machine. And yes I absolutely definitely want to have two times the same version of the code in different locations. Here the notion of the packaging systems paradigm of a “system” as a whole server and not a virtual host or any other subdivision of a system is in the way.
I prefer to use build systems like phing, ant or make to wrap up things. Especially phing in combination with DbDeploy is a very comfortable way to do things you don’t want to handle manually in the PHP universe.

Article
0 comment

Basics of Trend Scouting, Part 1

This will be the start of a short series of postings dealing with trend scouting or trend forecasting. The goal is to get an overview of things happening in trend scouting so I can link to my main area of interest which is technology scouting.
We will get straight some basic vocabulary, have a look on how trends propagate through society and what the lifetime cycle of a trend looks like. Most of this basic knowledge will help us to understand how technology scouting and trends in technology work. So lets start with some basic terms.

Trend

A trend can be any sort of physical or emotional shift or movement in society. A trend is a movement in a defined direction. A style (which is often confused with trends) in contrast is where you arrive when you follow a trend. In consequence a trend is a line or way, a style is a point or location. A trend is a sociological anomaly or oddity being visible as sort of a peak in an otherwise rather flat cultural landscape.

Trend scout / Trend forecaster

Trend scouts and forecasters are people looking for the afore mentioned patterns or shifts in attitude, lifestyle or mindset which contradict common thinking or behaviour.

The spreading of trends

Some trends spread in analogy to an infection. The phenomenon of this type of “cultural gene” is often referred to as a meme, a term coined by Richard Dawkins in his book The Selfish Gene.
The mechanism of proliferation can be discribed by a theory of Everett Rogers called “Diffusion of Innovations”. Based on data by Bryce Ryan and Neal Gross who observed how farmers in Iowa adapted innovations in farming technology he identified several groups of people, who adopt innovative changes with different speed. There are several diffusion of innovation curve variants, but the following image shows a common one:
Diffusion_of_Innovation
Innovators make up for 2.5% of all observed individuals, early adopters for around 13.5%. This shows that the fraction of people responsible for creating innovation and change is small. Its here where trend scouts and forecasters have to look for things coming.
In the next part we will have a look at how trends can be found and identified and how professional trend forecasters work.
To finish this first delivery I would like to list some literature references, which might be of interest:
Malcolm Gladwell, The Tipping Point: How Little Things Can Make a Big Difference
Martin Raymond, The Trend Forecaster’s Handbook