Notes: Work

My well-read friends sometimes ask what my favorite book is. And, damn it, I always feel a bit embarrassed answering, because out of all the books in the world, it’s Andy Weir’s The Martian.

Sure, it’s a great adventure novel, but there’s another reason. As a programmer, how could I not feel a bond with a guy who solved a pile of complicated engineering problems with duct tape?

Yes, of course duct tape works in a near vacuum. Duct tape works almost anywhere and everywhere. Duct tape is a gift from the gods, and we should worship it.

― Andy Weir, The Martian

We recently released another version of FirstBIT ERP, our business automation product for the UAE. A ton of work went into it, everything works as intended, there’s plenty to be proud of, and so on. I put a lot of effort into building a full data exchange with Bitrix24, for example, and I’m very happy I managed to finish it.

But deep down I’m still a little envious of our game development colleagues: our patch notes are interesting too, of course, but you won’t find anything like this in them.

A very interesting post about the trouble the “Clear All Rights” command can cause in a role. It does what it says, sure: the rights are gone. But the role object can end up eating several times more memory than it needs, out of nowhere.

I read it, thought about it, and went to check our configuration. And what do you know? I found a couple of dozen roles stuffed to the brim with “unchecked boxes.” The rights export weighed in at around three hundred megabytes. After a cleanup, it was almost three times smaller.

It’s curious that this had gone unnoticed for so long. Most likely, it’s down to the object count: our configuration is relatively small. Those roles just weren’t eating enough resources for us to notice.

A couple of weeks ago, we added an “Inactive” parameter to FirstBIT ERP’s product catalog. The task was simple: if a user no longer needs a product, they check the box and it disappears from everywhere, including selection forms and warehouse balance selection forms.

Sounds straightforward, right? The code was simple too: we went through every object where users select products and added an extra choice parameter.

But users started complaining that the setting didn’t work. We investigated and realized we’d forgotten input history. Turns out it just ignores choice parameters. Users selected a product in an invoice, marked it inactive, returned to the invoice, and… Saw the supposedly disabled product in the history again.

We looked for a way out. The problem is that input history lives in system storage, and there’s no way to change it from code. You can wipe it altogether, but it only actually clears when the client restarts, and even then it’s hit and miss. Disable history completely? That feels like curing a cold by cutting off the patient’s head.

At some point, we learned that input history is stored not just per field, but also per combination of choice parameters. Each combination of parameters and values has its own history. So, if we add another choice parameter to every product selection field, changing its value will “clear” the history. In reality, it creates a new one, but what difference does that make to the user?

So that’s what we did. It lives in shared storage and is passed to forms when they open through a common module that adds the choice parameter programmatically. When a user sets or clears Inactive for any product, the parameter changes, and already open forms receive it through notifications.

It all started so innocently, didn’t it?

It works pretty well, but the downsides are obvious. First, system storage gradually grows as new combinations of real choice parameters and our artificial one appear. Second, saving products is now a potential bottleneck: two users can’t save products with changed Inactive flags at the same time, because writing our hidden choice parameter’s new value to shared storage causes a lock.

The first problem can be solved by clearing storage on some trigger; the second, by storing our choice parameter per user, for example in an information register. Still, we’re really hoping 1C gives us access to input history before we have to pile more workarounds on top of the ones we’ve already built :-)

Infostart published an interesting article about the DBMS error “cannot insert duplicate key.” One possible cause is using an object’s unique identifier to match it during exchanges with other infobases, without a dedicated register mapping local objects to identifiers in the external database.

I haven’t yet encountered anything as wild as unique identifiers colliding across different infobases for the same table. But I abandoned the “single identifier” approach for good after building an exchange between FirstBIT ERP and GROTEM / Agent.

The first version worked like this: data from FirstBIT ERP was first exported to an intermediary 1C infobase written by GROTEM’s developers, and then to the mobile application server’s main database.

Matching was very simple: object identifiers. The same document in both databases had the same identifier, for example. But some mappings were more complicated: a FirstBIT ERP counterparty became three objects on GROTEM’s side: a counterparty, a retail outlet, and a settlement document.

Yes, a settlement document. All three objects had the same identifier, that of the FirstBIT ERP counterparty. It wasn’t particularly elegant, but the platform handles this arrangement perfectly well, and we decided there shouldn’t be any problems.

The exchange refused to work, though.

A quick look showed that the data made it to the intermediate infobase just fine and passed every integrity and completeness check. Only the last step was failing: sending it to the mobile application server’s database.

Eventually, it turned out that:

1. This database has a table storing all the objects we care about: documents, catalog items, and so forth.
2. That table has a unique index on the object GUID.

Well, mystery solved. Of course, the very first exported counterparty created three entities with the same GUID. They lived happily in their separate tables on the 1C side, but couldn’t be written into one shared table in the application database.

After upgrading the platform to 8.13.12.1790, we encountered an intermittent bug when switching a control’s current page on the server. It works on the server itself, but as soon as control returns to the client, the current page reverts to the previous one. The current-page-change event also fires, as though the user had changed it manually.

Unpleasant, but nothing terrible: we worked around the bug, documented it, and reported it to 1C. They’re still thinking about it. But weirdly, I’m a little disappointed. When you dig into these things, there’s always a bit of mystery. It feels like you’re about to uncover something interesting, some new quirk of the platform, maybe.

Then you finally look at the problem from the right angle, and it’s just an ordinary bug. Maybe striped or spotted, but a perfectly ordinary bug, sitting there twitching its antennae. And you think: damn, is that all I caught? :-)

When people talk about how something develops, a product, a language, a programming language, whatever, they sometimes call it “evolution.” The idea is that bad solutions gradually die out, while good ones stick around and make the thing better. Fair enough, but that’s not the part I’m thinking of right now. The catch is that descendants inherit more than the useful stuff. They also get every badly put-together, often pointless mechanism that didn’t quite manage to kill their ancestors.

As a result, we hold our breath while swallowing, get confused between dissatisfied and unsatisfactory, and program in the wonderful language JavaScript, which, handy and popular as it is, has picked up every flaw a scripting language could possibly have, and then some.

We’re still lucky, though: consider giraffes! Their laryngeal nerve first travels two meters down the neck and then back up to the laryngeal muscles, instead of going the few centimeters straight from the brain.

Knowing all this makes it a little easier to look at what my code sometimes turns into after another round of development. So what if this hamster has grown a dragon’s tail? Evolutionarily speaking, that’s a pretty useful tail. Who am I to argue with Darwin?

Like many companies, we give applicants a test assignment. It’s intentionally straightforward, mainly to see how well the candidate understands the platform and how carefully they work with configuration objects and code.

One little thing we look at is how they put together a long message with lots of parameters in it. A developer can use StrTemplate() or StrConcat(), or simply concatenate strings. We mark that last option down a little: the user sees the same result, but the code makes it much harder to picture roughly what the message will look like.

But there’s something else to it. In the first half of last year, Rarus published a report about its work for Beeline. There’s lots of interesting stuff in there, but one bit floored me: dropping string concatenation cut CPU load by 40%! They didn’t even need those two extra servers anymore.

The problem was that worker processes spent too much time allocating memory to concatenate strings. Joining two strings once isn’t going to hurt, but building a large query in a loop and doing it repeatedly can make the load snowball. Don’t use concatenation in those cases. The right tools are the same StrTemplate() or StrConcat(). On older platform versions without those methods, you can use a text document object: add lines to it, then retrieve the text.

A designer at Stack Overflow wrote about how they organize remote work. I liked the structure: it doesn’t just say “here’s how we do it,” but pairs each problem with a solution.

Almost all development at our company is remote, and I work remotely too. I really like working this way; the main thing is remembering how easy it is to pick up unexpected problems when your workplace and your home are the same place. Take the advice to drink water and get some sun: during a crunch, those things really are easy to forget.

Feedback is one of the coolest things in my work. It's not my first year of programming and I've written a lot of tools, mechanics and interfaces. However, ever complicated mechanism working as it should is… I don't know. It's like hearing “yes” on a graduation party, again and again :-)

It partially compensates another side of the coin. I mean a case when you are trying to understand one hundred levels of some application's abstraction for the hundredth time, again and again, through a lot of mistakes. Meanwhile, it's 2 a.m. already and coffee had replaced blood in your veins ages ago.

Ever since my colleagues and I redid the automation at KPAT, it has been running on Bus Terminal Management. It’s a 1C:Enterprise configuration, so hooking it up to pretty much anything is fairly quick.

First up, we wanted to send the bus schedules to E-Traffic and Yandex Schedules.

What Are These Services?

The first, E-Traffic, is Russia’s largest aggregator of bus terminal data. It collects information about scheduled departures and acts as an electronic ticket agent. Passengers don’t have to fight their way through individual bus terminal websites: E-Traffic lets them buy a ticket from anywhere to anywhere.

As for Yandex Schedules, it’s one of the Russian search engine’s side projects. Like E-Traffic, it’s mainly for passengers, and it can quickly put together fairly complicated routes with transfers. It doesn’t sell tickets, though.

How Did We Build the Export?

At first, we built a fairly elaborate setup to send schedules to both systems. We worked our asses off, until a conversation revealed that E-Traffic already knew how to pass schedules on to Yandex. Just like that, half the job disappeared.

On the technical side, we made a web service for Bus Terminal Management. E-Traffic sent a SOAP request, the service gathered the bus schedule data from the 1C:Enterprise infobase, and back came some XML.

We didn’t even have to write code to fetch schedules from every terminal in the KPAT network: all schedules are created and stored in one place, the main node of the distributed infobase, and that’s where we deployed the web service. In the end, coordination took longer than programming.

Where Can You See It?

The inner workings aren’t visible from outside, and there’s nothing unusual there anyway: robots talking to robots, big deal :-) The result of their chatter is much more interesting! For example, here’s today’s schedule from Kemerovo to Novokuznetsk.

An interesting post about terms and wording best avoided when designing interfaces. The aside about authentication and authorization amused me.

And, of course, the list looks incomplete without the famous contacts, and semiconductors :-)

The 1C:Enterprise configuration we implemented at bus terminals in Kuzbass a couple of years ago now has a nice new website.

I think it’s great! Partly because it’s one of those projects chipping away at a tired old myth about 1C that just won’t die, even among IT people: that it’s dull, sluggish software for accountants, while real programmers only write C++ (or Java, JavaScript, insert your choice here), and similar nonsense.

Vsevolod Ustinov wrote an interesting post about keeping a professional blog.

I’m not sure the format matters much, whether it’s a blog, an article, or a presentation for colleagues. But I completely agree with one of the main points: if you learn something but don’t write it down and chew it over a few more times, it’ll soon get washed away by your morning coffee.

A couple more useful links on the same subject:

• Why Keep a Blog (Despite Everything)
• A Programmer Doesn’t Have to Become an Influencer

Something occurred to me: long, hard, complicated things, studying, workouts, diets, all that misery, get a lot easier once you realize you really want what’s at the end of them.

So there you are, comfortable doing nothing, when you finally decide to lose some weight, or study for an MBA, whatever. Your blisters hurt, it all sucks, maybe you don’t need that MBA after all, and wouldn’t a lie-in beat a morning walk? But after the first go, a bit of studying, a jog, kefir instead of a pastry, you catch this quiet but ridiculously strong endorphin rush. Hey, look at me! I’m doing the right thing. I’m getting somewhere!

And when you slip up, playing a shooter instead of reading, eating cookies instead of cottage cheese, somehow never having time for the gym, you miss that kick. You eat cookies today and can’t stand looking at them for a week. Not because you don’t fancy them, but because you’re back to going nowhere, like a lump.

Getting somewhere really is a drug, as long as you know where you’re going and know you want to get there.

Reading my own code from last year has a distinct flavor of masochism. Ah, here I put an implicit query inside a loop, and here I retrieved a balance table and grouped it for some reason. Well done, buddy!

Still, at least it means I’m getting better. Say what you like about games, but sometimes I really wish real life had a pillar of light from the heavens whenever you leveled up.

Looking around inside the ticket offices at Belovo bus terminal took me straight back to that ancient DOS hospital sim, Theme Hospital.

At least, when I played it, I set up my GP offices in exactly the same way: identical square rooms, bin always in one corner, plant always in the other, radiator always under the window, and so on :-)