BeckDesignRules for Database Developers Part 1: Passes the tests

Software design and software architecture are topics which are sometimes associated with distant, slightly mad geniuses, living in an ivory tower and regularly throwing UML diagrams to their ground crew of “normal” developers. They are seen as arcane knowledge which can only be absorbed by the most experienced, most talented “10x developers“, “rockstars” and “ninjas”.

From my experience, this point of view couldn’t be farther from reality. On the contrary I see software design and architecture as fundamental parts of any kind of development work. The moment we start writing code, we will make decisions and are creating software design, no matter if we are aware or not.
Of course there might be guidelines, there might be an architecture given by others, there might even be detailed API descriptions and an abstract design which paints the big picture, but in the end the little, daily decisions will have a huge impact on the result.

Understanding the rules of good software design is therefore a topic which affects developers of any skill and experience level.
Knowledge in that field will be extremely beneficial for every task related to software development.

Four Rules of Simple Design

Kent Beck, developer of Extreme Programming (XP) and Test-Driven Development (TDD), came up with four rules of simple software design in the late 1990s, which Martin Fowler expresses like this:

BeckDesignRules

Martin gives a great, comprehensible description in his article about the “BeckDesignRules” which is also the source of the picture above.

I will try to take a deep dive into these rules, their spirit and how they can be practically done in database development. There is also much personal interpretation from my side and I’m not entirely sure whether Kent Beck would agree on all of my thoughts (of course I hope so).

I will split this up into several blog posts (you wouldn’t believe how hard it is anyway to keep my personal goal of one post per month) and today’s post will cover the first, probably most important and impactful of Beck’s design rules:

Passes the Tests

This one is clearly about Unit-Testing, Test-Driven Development and lots of green lights in your testing report, isn’t it? It means that good software design must adopt specific patterns from XP or other agile methodologies, right?
Continue reading

Advertisements

“You’re a panelist for 2018 CodeTalk Series” – about the struggles of giving a live presentation in a foreign language

 

It was a strange moment when Jacek, co-author and maintainer of utPLSQL, asked me last November if I’d like to be part of the next ODTUG CodeTalk series about PL/SQL testing. Not only that I immediately got hit by a strong impostor drive-by and felt way too low-skilled and inexperienced, the host would be Steven Feuerstein, Mr. PL/SQL himself and to that day an unreachable, a bit unreal celebrity for me.

Nonetheless I immediately agreed and tried to behave and communicate as professional and skillful as possible so I wouldn’t embarrass myself in front of the other, highly skilled and seasoned panelists of the talk.

While a good portion of tension and anxiety never faded completely I was very eager and motivated. It was also a great benefit to have Jacek continually encouraging me. Together with him and Pavel I developed a narrative and overall-theme for the talk, prepared a dedicated Star Wars demo-project which could be used for all the different parts of the CodeTalk and also did some slides.
It was fun and I got somewhat confident. I had that picture of myself in mind where I casually type code and explain in an entertaining and understandable way what I’m doing and why it’s important, switching to the prepared slides at some crucial situations. I had already done presentations in the past about technical topics in German which I think were good or at least okay. I knew what I wanted to present, built all the examples myself and therefore felt well-prepared.

We had the first meeting of all participants roughly a week before the event. I suggested to do a dry-run of my part and was encouraged to do so, while Steven would check how much time I’d need.

It was a disaster.

I couldn’t articulate one straight sentence, I mistyped the most basic statements, was constantly stammering and forgot lots of what I wanted to say, replacing it with meaningless jabbering or silence while trying to type.
To make things worse Steven told me that I just used 15 minutes when I had fought myself through about a third of what I wanted to show. I had previously estimated that my part would only take 10 minutes.

I guess it would have been much worse if the Steven, Jacek, Pavel and Stefan wouldn’t have been as kind, positive and encouraging as they were. I got lots of good advice, nonetheless I felt like I’d make them look ridiculous and they knew it.
At that point I knew very well that I had to change my approach completely.

Continue reading

Open Source engagement for utPLSQL

When I started to blog, I set myself the goal to publish one article per month. Maybe it was too ambitious, maybe I’ll have to learn to write shorter blog posts (which might not be possible because the reason I blog is to get a deeper understanding of a topic and reflect on it – something hardly doable with just scratching the surface).

Fact is I wasn’t able to publish for nearly 3 months and while I have some topics rumbling in my head I won’t just throw some unbaked thoughts on a blank page to satisfy a self-made deadline. Or grow a bigger audience.

This kind of little, personal posts is the only thing I’m willing to try (thanks Nate for your encouragement).
At the moment it feels a bit like writing a diary, but I will take it as an experiment.

The thing which kept me from writing for several weeks now is an amazing open source project named utPLSQL. It’s a unit-testing framework for PL/SQL and I started using it about a year ago when I needed a tool to help me getting a very much “grown” legacy project into a stable state.

After finding a bug around windows line breaks (yes, you can develop Oracle databases on Windows) and contributing a fix I had the pleasure to get in contact with the great guys working on the project. They were incredibly nice and helpful and so it was very natural to stay in contact.

Long story short – I finally ended up being the main contributor for the java-api and command-line-interface. It definitely helped that I could build upon a very good foundation and I learned an insane amount of new things which not only helped me in my job but also widened my horizon as developer.

Yesterday we successfully released version 3.1.0 of java-api and cli. I couldn’t include everything I wanted, but we pushed the project further and it was both fun and satisfying. There are some very positive effects of contributing to open source software I experienced, but that’s stuff for a different post.

To finally give this article at least a bit of technical value I will provide some short examples of the improvements of the new command-line-interface.

Continue reading

Are you sure about what you know?

“If you’re incompetent, you can’t know you’re incompetent … The skills you need to produce a right answer are exactly the skills you need to recognize what a right answer is.”

David Dunning [NYT]

Ever heard of the Dunning-Kruger-Effect? If not, you might want to change that – it’s a great scientific foundation to make fun of people who – for example – use buzzwords, obviously without knowing their meaning. It also gives you some academic ammunition when complaining about the technical incompetence of management, project lead or sales. By using the term and throwing it towards whoever you think is your opponent you can show your superiority. And isn’t it fun to insult people when they don’t even notice being insulted? Clearly a proof of your intellectual dominance, because you don’t fit into the group of incompetent people, do you? You are clever, reflected and open-minded enough not to fall to the traps of human psyche, aren’t you?

For those who don’t want to read through the linked Wikipedia-article, have this (very simplified) description:

The Dunning-Kruger-Effect describes a psychological phenomenon that people who are low-skilled in a certain topic are the most likely to think they know a lot and are skilled at least “above average”. This is due to the fact that their lack of knowledge on a subject also means they can’t recognize how much they don’t understand.
A low-skilled person can overcome these effects by education and increase of knowledge on the topic.
The other side of the effect is, that highly skilled people are the most likely to underestimate their own competence and tend to be less confident about their expertise.

dunning_kruger_effect

There’s a lot of information and commentary about the Dunning-Kruger-effect on the internet, often in combination with fingerpointing towards extreme examples of the phenomenon (you can read “Trump” between the lines here). Most of us have seen people whose lack of experience and knowledge on the subject they presented made us feel uneasy while they themselves seemed pretty confident and happy.
And I agree that it’s fun to watch the effect on others in its full glory, but if you calm down and think about it, it can suddenly become very serious (I guess that’s the reason why the work of Prof. Dunning and Kruger was awarded by the Ig Nobel Prize for “first making people laugh, then think”).

The problem is – this is not a phenomenon reserved for others. And it’s not something only present in ridiculous and extreme examples.

Continue reading

“One does not simply update a database” – migration based database development

1zlzpi

I already described in a previous post why databases are different to applications from a development perspective. The most impactful difference in my opinion is, that you can deploy applications by state. You pick a certain point in your version control system, build your application and have a certain state of your application at hand.

Best case, this state is backed by a number of self-tests and has a unique version number (well, it has one by the hash of your specific commit). You take that build and can deploy it, no matter if your customer had a different version (state) of your application installed or not.

Of course this is simplified, there might be external APIs, libraries, configuration files or – who had thought – databases which prevent you from easily switching application states, but normally such constraints are relatively few.

Databases are different. You can’t preserve state in a database once it’s shipped to a customer. The moment you give it out of your hands, a database starts to change its state continuously. You cannot simply “install” a new version of your database schema on a customer database – there’s data to be preserved and in most cases it’s not even possible to clearly separate schema and data.
And yes, data changes for applications, too, but in my experience database schema is usually much tighter coupled to the data than applications are (which makes sense, doesn’t it?).

So, how do we overcome these difficulties, especially if we are aiming towards frequent releases and agile, collaborative development? Continue reading

Strongholds of confidence: self-testing your database

Software-Testing has always been a very important topic in professional development because software – unlike for example buildings – changes a lot from the first line of code to release and from there during its whole lifetime. Due to that constant change and its complexity every honest developer will have to accept someday, that there is no software without bugs.

The rise of agile methods, ideas like continuous integration and the urge to have more frequent releases led to an even stronger focus on modern testing-strategies (interesting blog post about correlation of release frequency and agile practices – especially including automated testing).

Unfortunately there is some kind of “holy war” about some programming techniques around testing (I will write a bit more on that in the last chapter of this blog post), but nonetheless the need for automated self-testing (be it system-, integration- or unit-testing) in todays software development is real and uncontroversial among the experts. Therefore I will use the general term of “self-testing” in this article, concentrating on the very basic idea and trying to give an introduction on how it can be done in database projects. In my impression still many of today’s database projects don’t even know about the basics of self-testing, so let me try to show you the benefits without nit-picking about specific techniques.

Continue reading

There is no clean (database) development without Version Control

I was surprised when I did a bit of research for this current blog article and learned that SCCS, the first system for Version Control, was invented in 1972. That was 45 years ago. I wasn’t even born.

But even today, after such a long time, there are projects – especially database projects – which are not under version control. I myself remember several projects I worked on (or even was responsible for, shame on me) which were not under Version Control at all or at least the database part wasn’t. So why is this and what can we do against it?

Continue reading

DRY SQL – Writing simple, maintainable queries

One of the strongest recommendations for clean and pragmatic development is the DRY-principle:

Don’t Repeat Yourself!

This principle is not only part of every guide about professional development but also the reason why we even have things like subroutines, config files and  – hello, database professional – data normalization.

Although normalization should be a well-known practice for database professionals, I caught myself writing the same SQL statements again and again, copy-pasting parts as sub-selects, use it in procedures and functions and ending up at having the same query logic in many different places.

We database people are very strict about redundant data in our database design – why aren’t we as strict about the code we write?

Continue reading