Wednesday, 26 January 2022

The future (is now)

Quick post...

In a previous post, I talked about how I thought there would be a greater lock down of society and people would object.

While I'm not sure a greater lock down has occurred, I have noticed that media and self-styled elites are further creating a divide between you-know-who and you-know-who-not. Divide and conquer is a strategy as old as time. Unfortunately, most people are unsophisticated thinkers and fall victim to the game, handing the elite power on a platter. I'm sure I have done the same many times.

In that post I said:

There will be an attempt to wholly lock down society worse than what we've had the past two years. It will fail. It will be messy. It will not work. People will refuse it. People who matter to the smooth running of the economy, which is after all, anything anyone cares about these days.

The cultural elites driving this lock down will be caught off guard as they were caught off guard by he-who-shall-not-be-named. As with anyone losing power, they will try to increase their grip. It will be bad.

But in the end, the people will refuse to participate and will develop the leverage needed to add the right checks and balances to a system that is sorely lacking them. I doubt it will be violent.

We are at that point now where there is a mass trucker protest which they basically can't ignore. They tried, but it is too large now. The truckers matter to the smooth running of the economy. If there are going to be shortages and even worse inflation as a result of this strike, that is the people exercising their power against the government in a nonviolent manner. They can no longer blame a boogieman, even if state run news (lol) refuses to cover it.

The only way governments can respond are as follows:

  1. Accede/respond to demands and begin good faith negotiations
  2. Ignore the demands. Usually leads to government being toppled.
  3. Demand an end to the protest and respond with violence if the protest does not end
Surely there are other possible responses. What does history say governments do when their people protest in a massive organized fashion?

There is a long history of (3) in all professed "developed" countries. One might even say the US government response to Jan 6 falls under (3).

So the Canadian government response will be interesting. Will the enforcers support the leaders if they require violence?

I'm not so sure.

In any case, it will be interesting.

Other countries have already been doing mass protests for a long time. France, in particular. But mass protests in France are like brunch on Sundays, it's not guaranteed, but it'll happen.

Whether you want to believe it or not, the past two years have been basically a gift to the self-congratulating elites of the world. I wish I had the clip handy, but there is a woman at the WEF saying something to the effect of:

The good news is: we, the elite, get along better than ever! The bad news is: no one in our countries trusts us.

She actually did call herself elite. I wish I was kidding.This is how they see themselves. How do they see us?



Tuesday, 26 October 2021

The future

This blog shows that I am often early on trends. It took me a long time to realize this was the case. I'm not necessarily wrong, just early. I also don't really prepare well for trends that aren't potentially disastrous. For example, Bitcoin. Had I known how prices are determined (hint: what someone is willing to pay), I would have made different decisions.

Other trends I saw coming:

  • Golden handcuffs: The first time I saw someone get fired I said that I will never put myself in a position where I rely on a job. As a result, I have built multiple streams of income which have lasted for decades at this point and every year I try to add one more, usually unsuccessfully.
  • Remote work: Somewhat tied to the first, I have insisted on remote work (for the most part) for over 10 years. This is mildly tied to the logic here (tl;dr: people hire other people like themselves, I am not like other people so I try to avoid being in that position.) My two favorite positions I left because of lack of remote work instituted remote work within a year after my departure. Only a year early although the two events were nearly a decade apart!
  • A segregation between the elites and the rest of society. In the 2000s, I noticed that the culture had shifted to propagandizing families against each other. Cultural elites know that at the end, legacy is what matters. I remarked to a friend that marriage will be a consideration only for cultural elites in the near future. A few years later, I came across a really great book which confirmed my suspicions in data.
So what am I seeing today? A societal rip that is being exaggerated by an increasingly irrelevant media desperate to retain its control over the narrative. For a simple example, consider the distribution of feelings of those you know regarding vaccination status. Extreme on both sides, and being promoted by all media. Meanwhile, talk to someone who is off grid, they really don't care. Yes they'll wear a mask when they go into town, and no they won't otherwise.

To maintain control and push forward The True Narrative, the Internet will necessarily need to be locked down. We already saw this with the banning of a US president from social media. In my opinion, this should have shocked everyone, regardless of their political affiliation and should have been vehemently opposed by everyone. However, due to the aforementioned rip, it was quite easy to find out where one stood politically based on how they perceived that event. After all, I don't like the guy so it's better if he doesn't get to speak. This has since spilled over to censoring of all sorts of factual information on platforms like YouTube.

I don't know if I see this getting better without a major loss.

So what is the trend that I am possibly early (or maybe even late) on today?

There will be an attempt to wholly lock down society worse than what we've had the past two years. It will fail. It will be messy. It will not work. People will refuse it. People who matter to the smooth running of the economy, which is after all, anything anyone cares about these days.

The cultural elites driving this lock down will be caught off guard as they were caught off guard by he-who-shall-not-be-named. As with anyone losing power, they will try to increase their grip. It will be bad.

But in the end, the people will refuse to participate and will develop the leverage needed to add the right checks and balances to a system that is sorely lacking them. I doubt it will be violent.

I want to be wrong, but I also don't want to be wrong because I want that new component of checks and balances.

Am I really doing anything to prepare? No, not really. I don't expect it to be disastrous. Plus, I'm probably already late.

Monday, 30 August 2021

If software engineering is in demand, why is it so hard to get a job?

Title taken from here: https://news.ycombinator.com/item?id=28355658 where there is lots of speculation about why job searching in software is so Kafka-esque.

Without fail, every job I have ever taken is one where I was not super qualified but the other people in the job were like me in some way. Either they were independently minded, immigrants, liked Lisp for whatever stupid reason or wanted to do great things with people who wanted to do great things.

So looking for a job, in software, is about finding people like you. You can be perfectly qualified for the position, but if it's a company full of Chadbros, they want other Chadbros, if it's a company of bronies (yes, I've seen one, it was hilarious, and I loved it), they want people who don't challenge their bronyhood.

Let's take a look at the "About us" screenshots from some recently "Who's hiring" from Hacker News. Note that I'm absolutely not picking on these people, just pointing out some observations.


Commonality here could be: peppy personality


Commonality here could be: fantastic hair

You can keep going and they will all pretty much appear to share some commonality.


For comparison, here is a picnic:


Commonality here could be: people who like to have fun

Notice the picnic have a different "kind of people", even though presumably they all love to see their kids have fun.

So why do tech companies end up looking like the above?

Very simply: they know "their" kind of people and know how to get along with "their" kind of people. Whether this is brony, chad, young, old, culture 1, culture 2, etc, that is what _all_ interviews are about. The actual questions asked in the interview do not matter beyond basic some competence.

This is why interviewing has become a job in itself. Some people think it's perfectly normal to have 5 rounds of interviews but what is really happening here is: "would you like to spend time with this person".

This is also why networking is the best way to get a job if you are looking. You are already "their" people.

If you go for an interview, your job is pretty much to find out if you are compatible. It's a date. You'll know you're in if they help you eagerly with whatever fizzbuzz question they give you.

Yes I know people will say "how dare you, our interview process is supremely objective", but to provide my own anecdata, I've never gotten a job on technical merits, and I've had more than one offer over the years from big tech. Why I didn't take them is another story.

Many years ago, I was interviewing (blind) with one of those "interview with us and you get a job at FAANG" shops. I nailed everything technically, to the point where they said I was the best they had seen (could have been stroking my ego, who knows). I go onsite, and it could have been a bad day, they didn't like my hair, but the chemistry that was there remotely, on the days before zoom, disappeared and even though I nailed their onsite technical interviews as well - implementing Tetris I think - somehow, there was a complete cold distance. Shit, I even did it in Clojure, a language I hadn't learned before that day. So it wasn't technical at all.

It took me a while to understand what actually happened. They were not my people and I was not theirs. In fairness, I am not most people's people. It is what it is.

So look for people who are like you along some axis, or try to find some commonality beyond just answering the questions if you really want a shot at that job.

I realize that adds yet another dimension but in my opinion, it's the most important one.

My honest advice if you are looking for a job is: enjoy the tech interview process. The questions that everyone complains about can be fun and an opportunity to learn, whether you get a job is not even relevant. It's basically luck of the draw in the end. And focus on first impressions, those matter quite a bit.

Friday, 30 July 2021

Y's vs non-Y's (the parable of the software consultant)

Client: How much do you think it will cost to get the project done?

Consultant: Probably $X in P periods of time

Client: $X is too high, I can get 4 Ys for $X.

Consultant: OK, go for it.

... 2 x P periods of time pass ...

Client: Hey... The Ys didn't work out and the project is almost a writeoff. Can you come onto the project?

Consultant: Since I need to undo the damage first, it will cost $(X+Z)

Client: OK.

(Based on many true stories, names have been changed to protect the innocent.)

In my experience, the reason this happens is because software is eating the world, but deep knowledge of how software is built is still locked up in the old guard, some people coming out of high-tech FAANG-like companies, as well as some independent developers.

The rest of the industry is focused on leftpad-gate, and arguing over Javascript front-end frameworks etc while the art of developing software is honed by very few.

The flip side of the coin is the Y's can actually do a great job when it comes to software that follows a beaten path. For example, a standard web app with a React front-end. This is why everyone wants to know "your stack". Y's beat out non-Y's 99/100 times here in my experience.

Many conflate the proficiency of Y's with their home stack with software development expertise. I have found that this is usually not the case. They are SDK developers (which is perfectly fine, quite lucrative and very useful, please don't get me wrong.) If you take them out of their stack, they are lost.

The proficiency with Y projects (which vastly outnumber non-Y projects) trick you into thinking that you can use a Y for a non-Y project. You cannot.

When you get beyond Y projects, building software to support what is coming down the line is near impossible for Y's since there is no way for them to create something that doesn't exist. Y's can use a stack, but they can't create a stack.

For non-Y's, they are usually not interested in standard CRUD apps with React front-ends, so they are willing to wait for interesting projects to come to them.

I think the Y's and non-Y's are complementary and there is no need to pick a better side.

I know non-Y's who wish they could be Y's and I know Y's who yearn to be non-Y but do not get the opportunity. Maybe their paths are chosen years in advance by some fluke of life and the results compounded without their active consent.

Wednesday, 3 August 2016

Achieving goals using TeamCity

As a small company of 1 + sub contractors, I have very little room or patience for doing things I don't need to do. The concept of being an architecture astronaut is not new to me, as I have both designed and coded myself into a corner in the past so I am acutely aware of the YAGNI philosophy.

A very important class of tasks fall into the bucket of "things I need to do, but manually." This might include:

  1. Updating a sales database
  2. Generating development metrics to guage project health
  3. Monitoring license count/usage, bumping the count if needed or "upselling" the end user
  4. Announcing releases
These are all real tasks that I need to do every day/week/month. All of these are manual. Slowly, but surely, I am replacing them with automated scripts. That's not really too interesting.

The interesting part is that I'm using TeamCity to create the workflows to support all of these. At a quick glance:

TeamCity: not just for building software

Some projects/configurations are hidden, but this gives you the general idea. I not only use TeamCity for building and testing the software I write, but I use it to export sales, update mailing lists, generating development metrics and even automatically creating blog posts/emails announcing new releases.

Why would I do this?

Basically, I am buying myself time. By putting in a little bit of work ahead of time, I save myself man-months of work in the future.

What this really allows me to do, and is the ultimate goal, is focus on talking with customers. To create conversations with customers, you have to give them something they can talk about in the first place.

So every thought process is geared towards getting the customer communicating their pleasure, or displeasure with software they use.

Why is TeamCity the right tool to do this?

Well, for me it works because:
  1. I already use it and know it
  2. TeamCity has a very good concept of inputs and outputs: build artifacts. This is primarily what makes my non-build workflows possible
  3. My design process is agnostic of the tools/techniques I use, and so I can fit in whatever tool works best.
But before I choose the tool, I have a process for choosing an implementation. I've outlined the process below. I cannot take credit for this process as it was taught to me and I've honed it in the last little while.

The process

1. Choose a goal

Before I choose a tool to do the job, I look at the goal. While the high-level goal is obviously customer satisfaction (80% of people choose the "It's just perfect - thanks mom" option in recent surveys so I'm hitting that target pretty well), there are sub-goals that will help me get there. For an example, let's pick the "Notify customers of an update" goal.

2. Create a design

Next, I create a design to implement the goal. For example, here is the design I used to determine what happens when I commit a change. It's pretty standard though this evolved over the period of a few months. The purpose of creating this design is to let me go back to it and modify it when things change. Nothing is in my head, it's all here. This is a huge difference from only a couple of years ago where I would keep everything in my head.

3. Mark up the design where tools implement the required functionality

The next step is usually to print it out and circle places where I think tools will do the job I need. I simulated this with SnagIt as I only JUST recycled my actual notes for this particular diagram. Murphy's Law strikes again.

Why do this? Very simple: who wants to write a ton of code that isn't necessary? Outsource as much as possible to free (as in beer) software, commercial, whatever is needed to get you to your goal.

By doing things in this way, and keeping an eye on choosing things that let me do as little work as possible, I end up with a very lean design and most importantly, a lean implementation.

You can see the big arrow is the key piece here allowing me to outsource nearly 100% of the "notify users of an update" functionality. By linking in certain libraries on certain platforms and generating a very specific format of an update manifest, I can support updates on Windows, OSX and Linux.

In particular, Linux was super simple. Can you match the code below to the design above? It should be pretty straightforward.

cd $WLA_ROOT/build
export GNUPGHOME=$WLA_ROOT/dev/ci/keys/gpg
$WLA_ROOT/dev/scripts/run_gpg_command dpkg-sig -g " --yes" \
         --sign builder *.deb && \
    rm -rf repo && \
    mkdir repo && \
    cp *.deb repo && \

    cd repo && \
        (dpkg-scanpackages . /dev/null > Packages) && \
        (bzip2 -kf Packages) && \
        (apt-ftparchive release . > Release) && \
            ($WLA_ROOT/dev/scripts/run_gpg_command gpg --yes -abs -o Release.gpg Release) && \
            (gpg --export --armour > archive.key)

So... Why did you choose TeamCity when an actual workflow tool would do just fine?


Here is a version of the markup that makes it obvious where TeamCity can help: the package and manifest are treated as artifacts. The screenshot below, directly from TeamCity shows how treating the inputs and outputs as build artifacts allows the use of TeamCity as a tool for the rest of the process (after building and testing is complete):


Inputs for the "notify users of an update" box matching nearly 1:1 from design above. Linux has "15" because the "manifest" is a directory with a bunch of files in it so apt/yum  can do their job. Technically, I could zip that directory up and make it a single artifact.

What else do I get for free?

This is my favourite part of doing things like above: you get a ton for free. By integrating the everything into TeamCity, I get:
  1. Build history
  2. Artifact history
  3. Configuration management
  4. Project metrics (I specifically track LOC to make sure I'm not going too crazy):

Conclusion

I just wanted to record my thought process for posterity. Enjoy!

Wednesday, 1 June 2016

There is a very simple reason why the DAO was easily hacked

Background here. TL;DR: a smart contract "worth" about $150 million USD was "hacked" and had $45 million USD moved to accounts controlled by the hacker.

The "DAO" is an instance of "code as legal contract" run using a Bitcoin-like technology called Ethereum. The programming language for Ethereum is imperative for reasons that escape me, but are probably legit - likely simplicity and user friendliness.

There is a very simple reason why the DAO was easily hacked: the code is shit.

The contract (it doesn't matter if this is the deployed version or not) is about 300 lines long. It has the equivalent of 15 constants, 15 public fields, 2 nested classes and 25 functions.

25 functions all jumbled into one contract.

Would you let this pass code review? Most people would not. The organization of this contract was such that the quality of implementation was obvious from a first glance. The fact that this "hack" exists is not surprising. 25 operations on a type indicate that there is too much flexibility, and likely very little thought went into creating the correct abstractions. I say this from humbled experience.

As long as the Ethereum miners don't bail out the DAO, the concept of the Ethereum blockchain is fine. The DAO should fail to teach them a lesson. Bitcoin went through its own growing pains with all the exchange hacks forcing beefing up security.

There is an ongoing discussion on Hacker News about the code behind the contract.

All that being said, this incident should not turn people off Ethereum so long as the miners don't bail out the DAO. The technology is interesting, and this is the equivalent of the Araine 5 explosion for Ethereum. A learning experience.

Saturday, 6 December 2014

Why I moved to Git from Mercurial

There are stories everywhere about how Git is eating everyone's lunch. For a long time, I happily used Mercurial primarily because I needed a cross-platform DVCS that had a sane command-line interface. I don't think there is much disagreement that the first few iterations of git were not very portable nor was the command-line very friendly. Mercurial has been in daily use by me very nearly since it came out and I don't have any complaints. It's that good. So why move?

One word: Magit.

The only Emacs packages for Mercurial I tried (vc.el and Monky) were very basic. Monky, an imitation of Magit, was a very superficial approach and vc.el still hasn't moved very far out of the days of centralized version control, though it has somewhat. Additionally, Magit has a killer feature that rightfully matches Git's killer feature: editing the index.

ADHD Coding and the Index

Sometimes, due to what I assume is late-blooming ADHD, I go on development tangents where I fix bugs and/or add little features here and there. When it comes time to commit, I have many changes intermixed with one another and committing them all at the same time is a terrible idea for a clean, readable history.

Git's source editing process works like this: edit -> stage -> commit. Until you stage something, it cannot not be committed. Take a look at the Git book for more information.

What Magit allows you to do superbly (obviously through Git), is within each file, "stage" the only the changes you want. So if you have 3 blocks of changes within a file, you can choose to only stage the one change without reverting the other 2. Now you can commit related changes together even if you worked on multiple things at the same time.

So when using the ADHD development methodology described above, staging diff hunks is ridiculously helpful and Magit makes it easy to do the right thing even for simpletons: s for stage, u for unstage. I never did try very hard to figure out how to do it with Mercurial but I think I remember hearing the word "plugin" or "queues" before I zoned out.

Projects

I've moved most of my private repositories, including my JIRA time tracker over to Git as well as my dependency injection library for C++. Switching to Git for continuous builds was easy as pie. The only real change I had to make was the bit of the build that grabbed the HEAD revision:

diff --git a/ConfigureWorklogAssistant.cmake b/ConfigureWorklogAssistant.cmakeindex 988f6c4..1fb5d36 100644
--- a/ConfigureWorklogAssistant.cmake
+++ b/ConfigureWorklogAssistant.cmake
@@ -32,8 +32,9 @@ macro(define_kimi_product OutputFileVar ProductGUID CompanyName

   set(KIMI_PRODUCT_SOURCE_REVISION)
   execute_process(
-    COMMAND hg head tip --template "{rev}"
+    COMMAND git log -n 1 --format=%h
     OUTPUT_VARIABLE KIMI_PRODUCT_SOURCE_REVISION
+    OUTPUT_STRIP_TRAILING_WHITESPACE
     )

   configure_file(

The future

I am pretty much set with Git for now. The fact that it only took a few hours is probably a testament to how similar the two are. However, if Mercurial was to develop an interface as good as Magit, including supporting some form of staging in a big way, I think I would consider switching back. A DVCS written in Python is too delicious to ignore.