1 comment

Lean systems as reincarnation of large ones


Washington SunglassesSome software systems are designed for massive amounts of data to be processed in a very short time. Banking systems, fraud detection, billing systems. Lets pick one, I worked on for a long time: billing systems (for telecom or internet providers for example).

Most of those systems are very large, mostly complex systems, designed to bill millions of customers per month. Some examples are Kenan Arbor (bought by Lucent) or Amdocs. Since these systems need to process vast amounts of data very fast, they are built using compiled sources / binaries. Binary software is not easily customizable. So most of these systems are widely customizable via configuration files. Taking in account all options possible in dynamic configurations results in even more compiled code. I think you got it.

What actually is a billing system?

To give an impression which steps are required in a typical billing scenario, here is a short non-exhaustive list:

  1. Preprocessing
    Collect billable items from external systems, translate or reformat the data and put it in a database.
    Additional for mobile telecom billing: import GSM TAP3 (Transferred Account Procedure version 3) roaming data.
  2. First step: Rating
    Put a “price tag” on every billable item for the billing period in question
  3. Second step: Billing
    Collect rated items as invoice items per customer.
  4. Third step: Invoicing
    Create invoices, on paper or digitally.
  5. Fourth step: Payment
    Withdraw money via saved payment option per customer.
    Alternatively: substract invoice total from prepaid deposit.

The problem

All of these steps could be “special” for any customers. Think of a subscribed service. Every customer pays $5 per month. But once the company had an introductory offer of 20% off for the first year. So not only are some customers paying only $4 but they are paying $5 after 12 months. Now take into account, that a typical mobile telecom company has something like 20-30 different contract types or rates. Wow, lots of options. Not a problem for a multi-million dollar company but for smaller companies with, let’s say, 100 to 100.000 customers.

A solution?

Now what if a billing system would be implemented in a scripting language? Admittedly it would be a bit slower (would it? I don’t really know) that a solution in C or C++. But it would be very fast and flexible customizable, if well documented (we developers love to write documentation, don’t we?). Also management summary dashboards would be much more flexible as prebuilt solutions like QlikView (which also would cost additional license fees).

I could visualize for example a solution in Python. This way it would be fast compared to some other scripting languages and could leverage the massive amount of financial and mathematical software components. Build an administration and dashboard component with Flask or Django and run Python scripts on a PostgreSQL database. If more speed is needed you could switch to an Apache Spark architecture, which would also be scriptable in Python via PySpark.

Start on a small budget but don’t let decisions limit your options!

0 comment

Assholes and code of conduct manifests

Recently someone pointed out, that the perceived number of assholes is rising. This means, that anti-social behavior can be encountered more often. I responded that there seems to be some sort of sociological mechanism leading to anti-social behavior being more accepted than before.
Someone else asked what that mechanism might be, so here is my amateurish point of view.
Social interaction often is accomplished by communication. So examining forms of anti-social or disruptive communicational behavior might help to clarify some points. One form of disruptive communication is the so called “interactive vandalism” (Anthony Giddens, Sociology). Giddens points out, that effective communication or interaction is based on a cooperative behavior of the participants. If one party of an interaction deliberately behaves in a non-cooperative way, this often is encountered as a aggressive attitude by the other participants. But this is a stylistic device, not an explanation.
Another perspective is that of Erving Goffman‘s “The Presentation of Self in Everyday Life“: people behave as if they were acting. And like in a theater there is a stage and a backstage area. In the front region (stage) they act mostly according to common sense rules. In the back region (backstage) they can “give vent to feelings and styles of behavior they keep in check when on stage”. So acting more anti-social might mean transferring behavioral patterns from back to front.
In traditional social settings this would in general have been a completely unacceptable behavior, but why is it not judged that way now? There is a general process at work transferring the private into the public. Reality shows on TV, social media, liberalization of professional situations. Don’t get me wrong: I don’t judge those processes, but if not backed by a so called “good education” things can go wrong unnoticed. If this happens and some sort of “invisible control” doesn’t come into effect, openly visible regulation is a way to prevent unwanted situations.
An area where this currently happens are public conferences. More and more conference hosts issue code of conduct manifests. I got into some serious discussions because I objected that these rules are pretty obvious and stating them so explicitly might be sort of embarrassing for “well-behaved visitors”. Conference organizers assured me that these rules are not so obvious anymore. Maybe in the future we’ll see explicitly stated rules for human interaction more often. While being liberated from unnecessarily rigid forms of social behavior is a good thing, this feels like a cultural loss to me. But than again I just might be getting old.