Highload 2014

If you ever happened to hear about the biggest Russian IT conference Highload++, than you might be interested how it was. This is how it was for me this year: Highload 2014

I have deliberately chosen an external publishing platform for their beautiful storytelling format. Give it a try - Tilda.

Cheers.

Why REST is not JSON over HTTP

Because REST is distributed Hypermedia. THE END. 
Modern Web Architectures.

Enjoy.

Teach your code to speak genteelly, nil is country speach

Imagine you ask a person something, but he just stares at you in response. Or even worse, you tell one to do what you want, and he seems to get your command, but turns his back to you and you can't tell if he did what you asked him to or not. Did you imagine it?

Congratulations! You have just got nil/null, whatever represents nothingness in your programming language, in response to a function call.

The same way you think a person who doesn't react appropriately to your questions or requests is an ass-head, the same way function returning nil is a moron-function.

I do know that there are cases when you have nothing to return, but this is either a Special Case or an exceptional situation. Let's consider two examples for query and command. Yes, to be genteel your code have to comply with CQS principle. 

First - query. To be honest whether a special case or an exception is better depends very much on a context. Let's consider two query categories: query for data (information, data structure) and query for behaviour. First case is easy, you just fill your data structure with whatever it turns it to an empty state if it has a meaning or you raise an exception if you can't fulfill a query. Easy peasy. The second case is a little bit more complicated. Lets say, you write a polymorphic query function which is supposed to return objects with state and behaviour that may be used to do something. As long as it is polymorphic it is to change a behaviour when you work with descendant classes i.e. return different objects with different behaviour, but not every descendant alters a default behaviour. Oh, and there is no default behaviour. Natural choice? - Nil. But if I put it another way: to do nothing IS a default behaviour, it becomes obvious that your best bet is a Special Case with no op strategy, rather then nil, completely nothing. 

Now a command. I'd separate two kinds of commands: those that you do not expect to return a result and those you do. The first type is naturally implemented by procedures. Some programming languages have a void return type built in. This is probably the only valid case to return nothing. However, if your language of choice has no built-in tools to mark the function as a procedure then treat all commands as those that return result. In this case you'd probably return nothing in case you've failed to accomplish request, which is the worst way to do. Rather use an exception to inform a caller that his request puts you into an exceptional case when you can't process a command request.

All in all it is very easy to be polite to those who use your code and to those who read it. 

To be little bit more convincing I'll again reference one of the best programming books ever: Clean Code, and a very good discussion.

And as if that was not enough already I'd quote Sir Charles Antony Richard Hoare:

I call it my billion-dollar mistake. It was the invention of the null reference in 1965. [...] More recent programming languages like Spec# have introduced declarations for non-null references. This is the solution, which I rejected in 1965.

P.S. Should I mention that passing nil/null as a function argument is even worse for very obvious reasons? 

P.P.S. I'd like to try a programming language without nils. Without nothing. You got it.

Cheers.

TDD and Rails are sister pills

Давеча, небезызвестный в одном коммюнити человек DHH разразился замечательным высером про TDD, который по его мнению ничто иное как артефакт, подобранный на помойке истории, а применение оного напоминает ему онанизм с элементами BDSM.

Многое было сказано в защиту TDD, больше даже не в защиту, а так, поржать. 

Но ведь, если предположить, что Zed Shaw хотя бы наполовину прав, то сейчас вся толпа, которая разинув рот внимает божеству, бросится бросать и начнет начинать все сначала. 

Всем давно понятно, что OOP это лекарство чуть ли не хуже болезни, процесс принятия которого нужно сопровождать пилюлями типа TDD, BDD, DDD, чтобы пореже обкакиваться. Но у этих пилюль свои побочные эффекты и для них нужны другие пилюли типа autofixture (это вам .Net камарады подгон), DCI (а это скорее нам Ruby парням, обломитесь диназавры =Р).

Впрочем, если лекарство разбавлять, то все не так уж плохо. К примеру, у Udi Dahan есть много оригинальных идей, однако они относятся к разработке систем, а не приложений (разница я надеюсь понятна). А так ли нужен OOP для разработки приложений, а тем более TDD?
Может быть basecamp все же не все приложения/системы на свете, и надо быть более openminded, а DHH? Как-то даже странно это спрашивать у человека, который насрал на OOP в пользу developer productivity, ведь чтобы срать грамотно, надо быть openminded. 

Cознательно возведя productivity в краеугольный принцип Rails, DHH поставил его в одну категорию с TDD. Они суть две пилюли с разным действием, и вместе они не работают, т.к. мешают друг другу. 

Ну вот собственно и объяснение. Удачи.

Rails vs OOP or is it Rails feat. OOP?

Please mind that this note is as opinionated as can be. I believe this is the only way one can speak about Rails as RoR itself leave no room for doubts.

I believe Rails was meant for small to middle web sites development, and never meant to be a foundation for something bigger. Just read 37signals' Getting Real book and you will understand that Rails is backed by a company with a no-grow philosophy. And so is Rails - it will never be as big as Java or .Net are. 

You can ignore this until you start develop a little bit more elaborate software. And this is where you have to get serious about OOP. But Rails provides no aid here. Instead, it leads you into trouble, because it puts too much pressure on you with conventions and especially with poor naming. (I am not trying to charge DHH with that, after all naming is the weakest skill in IT community of all times.) Thousands of people fall into problems of not understanding where they should rethink Rails conventions and deviate from them, which leads to heated discussions whether Rails breaks OOP and how to keep producing quality maintainable and testable code with Rails (fat models/skinny controllers attempt, writing true unit-tests with Rails, is ActiveRecord an anti-pattern, etc). Well the answer is - get to know some OOP theory. These are the problems OOP community have been solving for decades now and there is nothing really new about that. 

Oh wait, there is something new! Rails deserves special gratitude for whetting community appetite for least known DCI design pattern. 

Rails community experience lot's of agitation and controversy, and this couldn't be another way. The reason for this is that RoR emphasizes on developers' productivity at the first place rather then being dogmatic about design. Understanding this may turn us to be a little bit less dogmatic about Rails' usage.

More to read:
https://gettingreal.37signals.com/
http://railsoopbook.com/
http://objectsonrails.com/
http://www.leansoftwarearchitecture.com/
http://clean-ruby.com/
http://www.artima.com/articles/dci_vision.html
http://folk.uio.no/trygver/2008/commonsense.pdf

Good luck!

TDD adventures Part 1. Productive TDD from real life.

Following is a real-life demonstration of how effective one may be applying TDD to solve tasks under stress conditions. 
Imagine that you are limited on time, you are under stress and you have a task you've never solved before. Did I mentioned that you don't know the problem domain and your destiny relies upon your solution you'll be judged for? How would you tackle this problem? What would you say If I'd tell you to write a test first? Sounds crazy, eh? You have so many things to do and you don't want to waste your time on testing. Afterwards, maybe... First you have to write... What are you going to write first? Remember, you aren't familiar with problem. Write a test.

Alright, now the problem statement is as follows:

"Compare two CIDR ranges. 

The significant part of IPv4 Address is prefix
255.255.255.255/24 notation means that only the first 24 bits of the IPv4 address make up the network mask. The bits which are not included in the network mask are not important.
There are several ways to write a CIDR range
These representations are equivalent:
255.255.255.255/24
255.255.255.0/24
255.255.255.100/24
Comparing CIDR Ranges

Ranges can be in subset, superset, disjoint or equal to each other. Write a code comparing them." 

- "Fuck, I never knew what this CIDR notation ment. I should have read about it before. Alright, let's google it. Whoooaaa! I'll read through this for half an hour. What do I do? I've already applied TDD in day to day job, I'll give it a try. First things first, I'll write down what I know about the problem. This is going to be my test list."

• CIDR range has prefix and suffix parts;
• Ranges with equal prefix and suffix are equal;
• Only the Prefix of the IPv4 Address is Significant;
• A CIDR Range Has Multiple Representations;

- "That will do for the moment. Which one of them do I pick first? I should pick the simplest to implement it as fast as possible. The first one is an ideal candidate for Starter Test."

    [TestFixture]
    public class CidrRangeTests
    {
        [Test]
        [TestCase("23.45.67.89/16", 23, 16)]
        public void when_instantiated_parses_string_into_address_prefix_and_suffix(string range, byte expectedMostSignificantByte, byte expectedSuffix)
        {
            //arrange 
            //act
            var sut = new CidrRange(range);

            //assert
            Assert.AreEqual(expectedMostSignificantByte, sut.Prefix[0]);
            Assert.AreEqual(expectedSuffix, sut.Suffix);
        }
    }

- "Now that the test is written I am going to write the CidrRange class itself. Forgive me Kent, but I am about to jump over one step - I won't Fake It, I consider this code to be an Obvious Implementation and 'll write it straight away. I had lot's of choclate and I can keep things in mind and I have very short time".

    public class CidrRange
    {
        //auto-props for the sake of simplicity
        public byte Suffix { get; private set; }
        public byte[] Prefix { get; private set; }

        public CidrRange(string range)
        {
            //assert not null
            var parts = range.Split('/');
            //assert count 2
            var prefixParts = parts[0].Split('.');
            //assert count 4
            Prefix = prefixParts.Select(byte.Parse).ToArray();
            Suffix = byte.Parse(parts[1]);
        }
     }

- "Green bar, hooray. Time spent? 10 minutes. OK, whats next? The next simplest test to implement - the shortest way to the green bar, is the second one: Ranges with equal prefix and suffix are equal".

        [Test]
        [TestCase("23.45.67.89/16")]
        [TestCase("1.2.3.4/24")]
        [TestCase("172.84.26.128/16")]
        [TestCase("197.54.16.128/25")]
        public void when_prefix_and_suffix_are_equal_should_consider_other_to_be_equal(string range)
        {
            //arrange 
            var sut = new CidrRange(range);
            var other = new CidrRange(range);

            //act
            //assert
            Assert.AreEqual(sut, other);
        }

- "And again, I triangulate and replace fake with implementation in one step. Remeber, I had lot's of chocolate."

        protected bool Equals(CidrRange other)
        {
            return Suffix == other.Suffix && Prefix.SequenceEquals(other.Prefix);
        }

- "OK, running the tests gives me a green bar. Yeah baby. What do I have to complete?"

• CIDR range has prefix and suffix parts;
• Ranges with equal prefix and suffix are equal;
• Only the Prefix of the IPv4 Address is Significant;
• A CIDR Range Has Multiple Representations;

- "OK, now stop and think. The last one seems to broaden the equality equation for CIDR range. If I put it another way it may sound like the following".

• when CIDR ranges with exact suffixes differ in insignificant bits, they are considered equal;

- "Cool, I feel for the core of the solution. Now I'll think more about the former: Only the Prefix of the IPv4 Address is Significant, - I may rephrase it like the following:

• when CIDR ranges with exact suffixes differ in significant bits, they aren't considered equal;

- "OK, so now my test list looks like this":

• CIDR range has prefix and suffix parts;
• Ranges with equal prefix and suffix are equal;
• when CIDR ranges with exact suffixes differ in insignificant bits, they are considered equal;
• when CIDR ranges with exact suffixes differ in significant bits, they aren't considered equal;

- "Let's code!"

        [Test]
        [TestCase("197.54.16.128/25", "198.54.16.128/25")]
        public void when_significant_prefix_bits_differ_should_not_consider_other_to_be_equal(string leftRange, string rightRange)
        {
            //arrange 
            var sut = new CidrRange(leftRange);
            var other = new CidrRange(rightRange);

            //act
            //assert
            Assert.AreNotEqual(sut, other);
        }

- "Red bar. OK, obviously in order to implement desired behavior I can no longer compare bytes, I have to drill down to bits. What facilities does BCL provides for this? I remember BitConverter and also there is a BitArray class. A-ha! This is what I am going to use, 'cause I don't really want to ressurect my middle age address arithmetic skills. The algorythm is simple: Given the CIDR range 255.255.255.255/24 we know that only the first 24 bits of the IPv4 address make up the network mask, - it means I have to take a prefix, turn it into a bit array, take a suffix, turn it into another bit array starting from the most significant bit and perform a bitwise AND operation in order to get a network mask. After I got both masks, I have to perform an operation, that will tell me if there any difference between them. Well, luckily I had lot's of choclate, and it's obvious that I need to perform a bitwise XOR".

        protected bool Equals(CidrRange other)
        {
            return Suffix == other.Suffix && PrefixesEquals(other);
        }

        private bool PrefixesEquals(CidrRange other)
        {
            var leftMask = new BitArray(Prefix);
            var leftSuffix = new BitArray(32, false);
            for (int i = 31; i >= Suffix; i--)
            {
                leftSuffix[i] = true;
            }
            leftMask = leftSuffix.And(leftMask);

            var rightMask = new BitArray(other.Prefix);
            var rightSuffix = new BitArray(32, false);
            for (int i = 31; i >= other.Suffix; i--)
            {
                rightSuffix[i] = true;
            }
            rightMask = rightSuffix.And(rightMask);

            var result = rightMask.Xor(leftMask);
            return result.Cast().ToArray().All(x => !x);
        }

- "Hmmm, red bar. Something is wrong here."
After five minutes in debugger it appeared that I was creating bit array for mask based on prefix with bytes in reversed order.

    public class CidrRange
    {
        //auto-props for the sake of simplicity
        public byte Suffix { get; private set; }
        public byte[] Prefix { get; private set; }

        public CidrRange(string range)
        {
            //assert not null
            var parts = range.Split('/');
            //assert count 2
            var prefixParts = parts[0].Split('.').Reverse();
            //assert count 4
            Prefix = prefixParts.Select(byte.Parse).ToArray();
            Suffix = byte.Parse(parts[1]);
        }

- "Arghhh, red bar. I'm toast. Come on, keep it on, I've almost found the solution and I've got tests that will prove it. The broken test is simple - Starter test expects to see bytes in reversed order, I'll fix it in a twinkle".

        [Test]
        [TestCase("23.45.67.89/16", 23, 16)]
        public void when_instantiated_parses_string_into_address_prefix_and_suffix(string range, byte expectedMostSignificantByte, byte expectedSuffix)
        {
            //arrange 
            //act
            var sut = new CidrRange(range);

            //assert
            Assert.AreEqual(expectedMostSignificantByte, sut.Prefix[3]);
            Assert.AreEqual(expectedSuffix, sut.Suffix);
        }

- "Finally! Green bar. It took me some time to get here, if I wasn't low on time I'd better be going teeny weeny steps of Fake and Triangulate. OK, now the test list looks like the following".

• CIDR range has prefix and suffix parts;
• Ranges with equal prefix and suffix are equal;
• when CIDR ranges with exact suffixes differ in insignificant bits, they are considered equal;
• when CIDR ranges with exact suffixes differ in significant bits, they aren't considered equal;

- "Let's code."

        [Test]
        [TestCase("23.45.67.89/16", "23.45.68.00/16")]
        public void when_insignificant_prefix_bits_differ_should_consider_other_to_be_equal(string leftRange, string rightRange)
        {
            //arrange 
            var sut = new CidrRange(leftRange);
            var other = new CidrRange(rightRange);

            //act
            //assert
            Assert.AreEqual(sut, other);
        }

- "red bar, eh? I was supposed to get a green bar!"

After another five minutes in debugger it appeared that I've filled the suffix mask with wrong number of bits and took into account the least significant bits comparing network masks. Fix follows.

        private bool PrefixesEquals(CidrRange other)
        {
            var leftMask = new BitArray(Prefix);
            var leftSuffix = new BitArray(32, false);
            for (int i = 31; i >= 32 - Suffix; i--)
            {
                leftSuffix[i] = true;
            }
            leftMask = leftSuffix.And(leftMask);

            var rightMask = new BitArray(other.Prefix);
            var rightSuffix = new BitArray(32, false);
            for (int i = 31; i >= 32 - other.Suffix; i--)
            {
                rightSuffix[i] = true;
            }
            rightMask = rightSuffix.And(rightMask);

            var result = rightMask.Xor(leftMask);
            return result.Cast().ToArray().Skip(32 - Math.Min(Suffix, other.Suffix)).All(x => !x);
        }

- "A wonderful, beautiful green bar! OK, what do I have?"

• CIDR range has prefix and suffix parts;
• Ranges with equal prefix and suffix are equal;
• when CIDR ranges with exact suffixes differ in insignificant bits, they are considered equal;
• when CIDR ranges with exact suffixes differ in significant bits, they aren't considered equal;

- "Well, this is not the end, but this was the most interesting part. Following is what is left".

• CIDR range has prefix and suffix parts;
• Ranges with equal prefix and suffix are equal;
• when CIDR ranges with exact suffixes differ in insignificant bits, they are considered equal;
• when CIDR ranges with exact suffixes differ in significant bits, they aren't considered equal;
• CIDR range is a subset of other range if prefixes are equal and the suffix is greater;
• CIDR range is a superset of other range if prefixes are equal and the suffix is less;
• CIDR range is a disjoint of other range if prefixes differ;

- "I'll take them one by one."

        [Test]
        [TestCase("1.2.3.4/24", "1.2.3.4/16")]
        public void when_prefixes_are_equal_and_left_suffix_is_greater_should_consider_left_to_be_a_subset(string leftRange, string rightRange)
        {
            //arrange 
            var sut = new CidrRange(leftRange);
            var right = new CidrRange(rightRange);

            //act
            var actual = sut.CompareTo(right);
            var expected = sut.Prefix.SequenceEqual(right.Prefix) && sut.Suffix > right.Suffix
                ? RangeIntersectionResult.Subset
                : RangeIntersectionResult.Disjoint;

            //assert
            Assert.AreEqual(expected, actual);
        }

- "Obviously there are more then two results of ranges comparison. There are four of them as follows".

    public enum RangeIntersectionResult
    {
        Equals,
        Subset,
        Superset,
        Disjoint
    }

- "And to compare my range objects I will need another method."

        public RangeIntersectionResult CompareTo(CidrRange other)
        {
            if (Equals(other))
                return RangeIntersectionResult.Equals;

            if (PrefixesEquals(other) && Suffix > other.Suffix)
                return RangeIntersectionResult.Subset;

            return RangeIntersectionResult.Disjoint;
        }

- "Green bar! Hooray."

• CIDR range has prefix and suffix parts;
• Ranges with equal prefix and suffix are equal;
• when CIDR ranges with exact suffixes differ in insignificant bits, they are considered equal;
• when CIDR ranges with exact suffixes differ in significant bits, they aren't considered equal;
• CIDR range is a subset of other range if prefixes are equal and the suffix is greater;
• CIDR range is a superset of other range if prefixes are equal and the suffix is less;
• CIDR range is a disjoint of other range if prefixes differ;

        [Test]
        [TestCase("172.84.26.128/16", "172.84.26.255/24")]
        public void when_prefixes_are_equal_and_left_suffix_is_less_should_consider_left_to_be_a_superset(string leftRange, string rightRange)
        {
            //arrange 
            var sut = new CidrRange(leftRange);
            var right = new CidrRange(rightRange);

            //act
            var actual = sut.CompareTo(right);
            var expected = sut.Prefix.Skip(1).SequenceEqual(right.Prefix.Skip(1)) && sut.Suffix < right.Suffix
                ? RangeIntersectionResult.Superset
                : RangeIntersectionResult.Disjoint;

            //assert
            Assert.AreEqual(expected, actual);
        }

        public RangeIntersectionResult CompareTo(CidrRange other)
        {
            if (Equals(other))
                return RangeIntersectionResult.Equals;

            if (PrefixesEquals(other) && Suffix > other.Suffix)
                return RangeIntersectionResult.Subset;
            if (PrefixesEquals(other) && Suffix < other.Suffix)
                return RangeIntersectionResult.Superset;

            return RangeIntersectionResult.Disjoint;
        }

• CIDR range has prefix and suffix parts;
• Ranges with equal prefix and suffix are equal;
• when CIDR ranges with exact suffixes differ in insignificant bits, they are considered equal;
• when CIDR ranges with exact suffixes differ in significant bits, they aren't considered equal;
• CIDR range is a subset of other range if prefixes are equal and the suffix is greater;
• CIDR range is a superset of other range if prefixes are equal and the suffix is less;
• CIDR range is a disjoint of other range if prefixes differ;

        [Test]
        [TestCase("197.54.16.128/25", "197.54.16.127/25")]
        [TestCase("205.00.00.1/32", "205.00.00.00/32")]
        public void when_prefixes_are_different_should_consider_left_to_be_a_disjoint(string leftRange, string rightRange)
        {
            //arrange 
            var sut = new CidrRange(leftRange);
            var right = new CidrRange(rightRange);

            //act
            var actual = sut.CompareTo(right);
            var expected = !sut.Prefix.SequenceEqual(right.Prefix)//this is not exactly correct specification, but it's OK for the sake of simplicity
                ? RangeIntersectionResult.Disjoint
                : RangeIntersectionResult.Subset;

            //assert
            Assert.AreEqual(expected, actual);
        }

• CIDR range has prefix and suffix parts;
• Ranges with equal prefix and suffix are equal;
• when CIDR ranges with exact suffixes differ in insignificant bits, they are considered equal;
• when CIDR ranges with exact suffixes differ in significant bits, they aren't considered equal;
• CIDR range is a subset of other range if prefixes are equal and the suffix is greater;
• CIDR range is a superset of other range if prefixes are equal and the suffix is less;
• CIDR range is a disjoint of other range if prefixes differ;

- "Time? Wow, I still got some time left. I've manged to complete in under two hours"

This is a real life story disproving the myth of TDD nonproductiveness. Stay tuned as I'll continue my adventures in TDD.

Next time I will refactor this code in a quiet and peacful manner.

P.S. You can find code here.

The Passionate Programmer book findings.

Here I would like to promote to everyone this excelent book.

The Passionate Programmer book tells how not to fall in despair doing your regular programmer's work and work out remarkable software developer career. All ideas presented in form of blog posts related to various areas of selfdevelopment.

This book is full of obvious things, actually. But these are the things you don't always remember about, seeing no forest for the trees.

Thus, I've decided to write'em down, mostly to memorize them better. You may also find these thoughts useful, but I would strongly suggest to come down with your money and buy this book.

It starts with a chapter named "Find your market". Following is a list of recipes how to choose a right market.

1. Leed or bleed? Both ends of the technology adoption curve might prove to be lucrative.
2. Supply and Demand. You can’t compete on price. Exploit market imbalances.
3. Coding don't cut it anymore. Just like technologies that become hot, business domains can be selected in the same way.
4. Be the worst. This one is my favorite. In short you should always try to find people that are better then you and work with them. The people around you affect your own performance. Choose your crowd wisely.
5. Invest in your intelligence. Love to learn new things.
6. Don't listen to your parents. Take calculated riskswith your career.Don’t let fear consume you.
7. Be a Generalist. Generalists are rare...and, therefore, precious. Your skills should transcend technology platforms.
8. Be a Specialist. Specializing in something doesn't means not knowing about other things.
9. Don't put all your eggs in someone else's basket. Though it’s possible to build a business that exists as a parasite of another, as an individual it’s an incredibly risky thing to do. Vendor-centric views are typically myopic.