вторник, 17 марта 2015 г.

Как стать хорошим программистом

Зададимся вопросом: что нужно для того, чтобы стать хорошим программистом?

Как и везде, нужно развивать качества и навыки.

Для разработчика есть несколько навыков, которые, фактически, обязательны.

  1.     Уметь читать и писать по-английски
  2.     Уметь "вслепую" печатать на клавиатуре
  3.     Уметь грамотно и связно выражаться

Со стороны качеств - нужно всегда осознавать, что каждый день придётся узнавать что-то новое, процесс обучения не останавливается никогда. Нужно также в целом любить читать.

Со своего опыта могу сказать, что только с такими качествами можно нормально ориентироваться в разработке ПО.

Есть несколько книг, которые реально полезны для начинающего разработчика, и все они переведены на русский.

И да, математика в общем не нужна.

Так как вопрос о том, как стать хорошим программистом, то я делаю допущение о том, что вопроса "какие именно программы разрабатывать" не стоит. Нужно осознавать, что в целом можно выделить четыре типа приложений, и гораздо проще будет "осесть" в каком-то одном, а не пытаться уметь на мастерском уровне делать всё (это, на мой взгляд, не достижимо).

  1. Веб-приложения, такие, как, например, этот блог.
  2. Приложения реального времени, такие, как, например, серьёзные симуляции виртуальной реальности ("компьютерные игры"), или Google Earth.
  3. Программное обеспечение контроллеров, встроенное в аппаратуру ("встроенное ПО"), например, ПО калькулятора, стиральной машины, бионического протеза, инсулиновой помпы.
  4. "Коробочное" ПО, традиционное, которое условно "устанавливается" на компьютер и позволяет решать какие-то задачи.

Это разделение лично моё, в разных источниках можно прочитать другие варианты. Суть в том, что разные программы выглядят, составляются, и обслуживаются по-разному. Можно ещё условно выделить мобильные приложения, которые суть нечто среднее между веб-приложениями и "коробочными".

Я обещал книги. Вот они:

  1. "Чистый код", Роберт Мартин
  2. "Совершенный код", Стив Макконнелл
  3. "Программист-прагматик. Путь от подмастерья к мастеру", Эндрю Хант et al.
  4. "Идеальный программист", Роберт Мартин
  5. "Алгоритмы. Руководство по разработке", Стивен Скиена
  6. "Человеческий фактор. Успешные проекты и команды", Том ДеМарко et al.

Здесь перечислены книги, которые образуют фундамент знания разработчика любого ПО вообще. Они великолепны. Я вот "алгоритмы" ещё сам не дочитал, но это потому что она просто систематизирует то, что я уже на собственном опыте узнал из разных источников. После этих книг имеет смысл почитать хардкор:

  1. "Структура и Интерпретация Компьютерных Программ", Харольд Абельсон et al.
  2. На русском нет: "Concepts, Techniques, and Models of Computer Programming", Peter Van Roy

Есть ещё несколько книг такого же характера - углубляющих знания и описывающих хитрые неочевидные приёмы, но я их ещё не читал, поэтому рекомендовать не буду.

Ещё есть пять книг, описывающих приемы не разработки, а сопровождения программ.

  1. "Рефакторинг", Мартин Фаулер et al.
  2. "Приемы объектно-ориентированного проектирования. Паттерны проектирования", Эрих Гамма et al.
  3. "Шаблоны корпоративных приложений", Мартин Фаулер et al.
  4. На русском нет: "Specification by Example", Gojko Adzic
  5. "Непрерывное развертывание ПО", Джез Хамбл
Весь этот свод из 13 книг - это Веды разработки ПО. Никто не скажет, как стать "хорошим программистом" лучше, чем они.

Есть, конечно же, второй способ, более лёгкий, но который нельзя пройти в одиночку. ;) Можно найти уже установившегося "хорошего программиста" и работать с ним в паре. Есть техника "парного программирования", но я не об этом, там нужно, чтобы квалификация была одного уровня. Я о том, чтобы делать за него какую-нибудь вспомогательную работу, и параллельно учиться на его примере.

Помимо фундаментальных знаний, нужны навыки собственно программирования, решения задач составлением программ.

Для их развития я лично бы порекомендовал выбрать какой-то один язык программирования и начать на нём решать какие-нибудь задачи, для начала простенькие.

Например, вы решили стать разработчиком веб-приложений. Освойте платформу Ruby on Rails, пройдя последовательно вот это руководство: http://railstutorial.ru/chapters/4_0/beginning или платформу Node.js, начав отсюда: https://nodejs.org/documentation/tutorials/ (я предупреждал про английский язык), или платформу PHP, начав отсюда: http://php.net/manual/ru/ (я предупреждал про любовь к чтению). Это в целом равномощные платформы, разница только в синтаксисе и некоторых специфических возможностях, трудно реализуемых на смежных платформах.

Или вы решили писать встроенное ПО. Тогда стоит начать с проекта Arduino: http://www.arduino.cc/, правда придётся вложиться в "железо".

Или вы решили писать "коробочное" ПО, тут у вас огромный выбор языков программирования, например, Haskell: https://www.haskell.org/ или Erlang: http://learnyousomeerlang.com/ или C#: https://msdn.microsoft.com/ru-ru/library/67ef8sbd.aspx или Common Lisp: http://www.gigamonkeys.com/book/

Кстати, для оценки "хорошести" программиста можно воспользоваться вот этой "матрицей компетентности программиста": http://dev.by/pages/programming_matrix

вторник, 21 октября 2014 г.

Web Application Development with Yii 2 and PHP book (by me)

Holy cow, I wrote a book!

After the terror of 6 months writing, re-reading and re-writing, "Web Application Development with Yii 2 and PHP" have finally been published at 26th of September.

It was completely unexpected for me to receive this contract, I have never used version 2 of Yii framework, which was in early beta at the time, and had quite small Web presence to be spotted by any publisher. But it happened (thank you, God).

Yeah, URL is https://www.packtpub.com/web-development/web-application-development-yii-2-and-php, and, honestly, without any self-promotion, if you want to develop using Yii 2, go buy this book, because I took a lot of time inspecting this framework so you will not need to do the same. For example, you will not get the exact details of how the error reporting works in Yii from the official documentation. The code is always the last source of truth, and I tried hard to express in plain technical English what it actually does.

To be honest, I strongly dislike Yii. I think that Yii 2 is enormous leap forward since version 1.1.x but I still dislike this framework. Each book I read about pro-level software development tells me things which I find completely ignored or misinterpreted in Yii. Reading its code was also not a pleasurable experience.

And trust me, our team at Clevertech is maintaining a ~80KLOC application based on Yii 1.1 idioms (initially) and at second year we came to conclusion that we don't need most of it. For example the whole ActiveRecord layer. I don't even talk about basing the project structure on the conventions, "models-controllers-views" directory triad is a biggest bullshit I saw so far in my career. Our "common/models" directory has like 15 subfolders in it, and we have no other choice, because you cannot stuff everything into ActiveRecord descendant, and there's no "domain_models" directory in the bastard of MVC which Yii implements.

Err... I went away. The book. Right.

I wrote it nevertheless. I am working with Yii application (version 1.1 though) every day and actually can tell a thing or two about it. So, publisher asked and I answered.

I had two main premises when writing.

  1. Show how you can use Yii 2 when developing your application, not simply "how to use Yii 2", which is covered already by official documentation.
  2. Show how to work in ATDD manner with CI in mind, even using the framework. There is clearly not enough literature which shows how to properly develop applications right from the start, without rudimentary examples like bowling kata.
So, the whole Chapter 2 is a glorious implementation of the point 2. Or so I thought. This chapter ended being probably of the worst quality from the whole book. It contains so much testing harness and deployment machine setup that in the end I separated the whole Appendix out of it and had thrown away around 10 pages which were not related to Yii 2 at all. And it  may be still so boring to read (if you wanted to learn Yii 2 and know that stuff already) that probably 9/10 readers will drop the book right there and not go to the really interesting stuff like how Yii 2 extensions work or its event system or new Request/Response model.

The book uses a single example application, slowly adding more and more features from chapter to chapter. The application is pretty trivial, it's a CRM skeleton, and by the end of the book there are mainly two features: adding the customer and fetching his info given the phone number; and sign in to the system using login/password. There are also a whole Yii 2 extension, though.

I seriously tried to put all I learned through the professional literature like the DDD, GOOS, Clean Code and such, but I personally think I failed at this quite spectacularly. :) But apart from that, I tried my best to show the pieces of the Yii 2 picture which are not covered (and probably will never be) by the official docs and references. Hope it'll be useful for you.

Also, there's no other book on this topic yet so you have no choice anyway. ;)

There were a serious misunderstanding from my part also: I completely missed the point that this book is legally the "second edition" of the previous book, "Web Application Development with Yii and PHP" (note the absence of the number 2) by different author, Jeffrey Winesett. The outcome of this is that I become something like a co-author with both names present on the cover. It hurt a bit, as I lived 6 months in permanent crunch mode (book writing is not my fulltime job after all) and Jeff is not related to this book at all, but we both are listed as authors equally.
This does not apply to the royalties, though, so I am not bothered by this so much.

I did this for greed, not for pride, after all. :) This is not the kind of book I would be proud of. Personally, I would promote Common Lisp-based development, not the Yii 2 framework (doh) over PHP (holy cow).

The (not so great) Return

Well, no luck with Logdown. Call me greedy for fame, but here in Blogger, integrated with Google+, I have a lot larger audience coverage than at Logdown.

Also, I am shifting from Markdown to AsciiDoc, so probably generated HTML will be more Blogger-friendly (I think "no way" though).

So, in next several days (probably) I'll return all my posts written at Logdown to here (there not so many anyway) and will continue to write here.

By the way, I have written and published a book with Packt. I'll write about it in another post.



Нафиг Logdown. Называйте меня жадным до славы, но здесь на блоггере с интеграцией в гуглоплюс у меня больше покрытие чем там (какого хрена, мои посты даже в гугле не проиндексированы).

Также, я теперь пишу не в Markdown а на AsciiDoc, так что, возможно, сгенерированный HTML будет более съедобным для блоггера (хера с два, я подозреваю).

Так что через насколько дней я всё написанное на Logdown мигрирую обратно сюда и продолжу писать здесь.

Между прочим, я написал и опубликовал книгу с Packt Publishing. Напишу об этом в другом посте.

вторник, 17 сентября 2013 г.

Переезжаю с Blogger на Logdown / Migrating from Blogger to Logdown

Ну что ж, пора мучений с HTML-разметкой закончилась. Нашёл блогоплатформу Logdown, там можно писать в github-flavored markdown, плюс подсветка кода встроенная. Плюс встроенные шаблоны просто чудесные, хотя шрифтам явно не хватает поддержки кириллицы.

В этом блоге больше писать не буду, буду в logdown теперь. Все старые посты отсюда перенёс туда (разметка, конечно, вся поехала, придётся чинить). Правда, этот блог удалять не буду, мало ли у кого постоянные ссылки сюда есть.

Новый адрес тут: hijarian.logdown.com (yeah, this is my new address, my dear English-speaking readers).

суббота, 20 июля 2013 г.

How to package and use Yii framework in PHAR archive

Okay, today's the task: pushing all of 1892 files of Yii framework to the Git repository is a burden, and upgrading it to new version pollutes the git log with changes to files you don't care about. Let's package it into a PHAR!

Packaging

Rasmus Schultz made a special script to package the Yii framework into a PHAR archive. I forked it to save for a future (at least my GitHub account will live as long as this blog).

You need just to put this script to the root of Yii codebase (cloned github repo, for example), and run it as usual:

php yii-phar.php

This will create the PHAR archive in the same directory.

Hovewer, beware the catch 1: PHP can refuse to create packed archive, emitting the following error:

PHP Fatal error:  Uncaught exception 'BadMethodCallException' 
with message 'unable to create temporary file' 
in /path/to/your/yii/root/yii-phar.php:142
Stack trace:
#0 /path/to/your/yii/root/yii-phar.php(142): Phar->compressFiles(4096)
#1 {main}
  thrown in /parh/to/your/yii/root/yii-phar.php on line 142

I decided to just remove lines 140 and 142 from the script:

echo "Compressing files ...\n\n";

$phar->compressFiles($mode);

And that's all. I can bear with 20 MB file in repo, and don't really care about compression.

Using

To connect the resulting PHAR to your Yii application, replace your usual:

require_once('/path/to/your/yii/framework/yii.php');

With the following:

new Phar('/path/to/yii.phar');
require_once('phar://yii/yii.php');

Note that in new Phar() invocation you should use real path to your phar archive file, but second line should be written verbatim, as the PHAR which becomes created is being made with alias 'yii', using feature described in the documentation for Phar::__construct.

However, of course, there's a catch 2: Yii built-in asset manager (CAssetManager) has too specific `publish` method, unable to cope with custom PHP streams. So, we need the fixed version.

I decided to create a descendant of `CAssetManager` descriptively called `PharCompatibleAssetManager` with the following definition exactly:

/**
 * Class PharCompatibleAssetManager
 *
 * As we use Yii packaged into .phar archive, we need to make changes into the Asset Manager,
 * according to the https://code.google.com/p/yii/issues/detail?id=3104
 *
 * Details about packaging Yii into .phar archive can be found at
 * https://gist.github.com/mindplay-dk/1607318
 */
class PharCompatibleAssetManager extends CAssetManager
{
  protected $_published = array();

  public function publish($path,$hashByName=false,$level=-1,$forceCopy=null)
  {
    if($forceCopy===null)
      $forceCopy=$this->forceCopy;
    if($forceCopy && $this->linkAssets)
      throw new CException(Yii::t('yii','The "forceCopy" and "linkAssets" cannot be both true.'));
    if(isset($this->_published[$path]))
      return $this->_published[$path];

    $isPhar = strncmp('phar://', $path, 7) === 0;
    $src = $isPhar ? $path : realpath($path);

    if ($isPhar && $this->linkAssets)
    {
      throw new CException(
        Yii::t(
          'yii',
          'The asset "{asset}" cannot be published using symlink, because the file resides in a phar.',
          array('{asset}' => $path)
        )
      );
    }

    if ($src !== false || $isPhar)
    {
      $dir=$this->generatePath($src,$hashByName);
      $dstDir=$this->getBasePath().DIRECTORY_SEPARATOR.$dir;
      if(is_file($src))
      {
        $fileName=basename($src);
        $dstFile=$dstDir.DIRECTORY_SEPARATOR.$fileName;

        if(!is_dir($dstDir))
        {
          mkdir($dstDir,$this->newDirMode,true);
          @chmod($dstDir,$this->newDirMode);
        }

        if($this->linkAssets && !is_file($dstFile)) symlink($src,$dstFile);
        elseif(@filemtime($dstFile)<@filemtime($src))
        {
          copy($src,$dstFile);
          @chmod($dstFile,$this->newFileMode);
        }

        return $this->_published[$path]=$this->getBaseUrl()."/$dir/$fileName";
      }
      elseif(is_dir($src))
      {
        if($this->linkAssets && !is_dir($dstDir))
        {
          symlink($src,$dstDir);
        }
        elseif(!is_dir($dstDir) || $forceCopy)
        {
          CFileHelper::copyDirectory($src,$dstDir,array(
            'exclude'=>$this->excludeFiles,
            'level'=>$level,
            'newDirMode'=>$this->newDirMode,
            'newFileMode'=>$this->newFileMode,
          ));
        }

        return $this->_published[$path]=$this->getBaseUrl().'/'.$dir;
      }
    }
    throw new CException(Yii::t('yii','The asset "{asset}" to be published does not exist.',
      array('{asset}'=>$path)));
  }
}

I'm really, really sorry that you had to read this traditionally horrible Yii code, but that was inevitable... :(

Main change was starting from the $isPhar = strncmp('phar://', $path, 7) === 0; part.

Now just link this asset manager instead of built-in one:

config/main.php:
        'components' => array(
            'assetManager' => array(
                'class' => 'your.alias.path.to.PharCompatibleAssetManager',
            ),
        )

Congratulations!

Now your Yii web application uses phar archive instead of huge pile of separate files. They say that this increases performance, but my personal reasons was just to reduce the number of files inside the repository.

суббота, 29 июня 2013 г.

Running Database Dependent Tests on Ramdisk

Today's guess is this: you have a test harness which utilizes the database, and it has enough test cases in it for full test run to be so slow you cringe at the very thought of launching it.

We discuss a lifehack-style solution to this problem: putting the DBMS which will be used by our test harness completely on a RAM disk, so it'll operate from much faster memory than the hard drive (even faster than from SSD).

Main issue is this: as you probably need only a test datasets at ramdisk, and only for a period of running test suite, you will need separate DBMS instances to work on ramdisk, not the ones already installed on your system.

Here we'll look at how to prepare MySQL and MongoDB instances to work on ramdisk.

End Result

In the end you'll get the special preparation script which you should launch before your tests. After this, your test suite will run with the isolated MySQL and MongoDB instances on top of ramdisk.

If your test suite has large quantities of integration tests using databases, this will greatly increase the speed of test run. It is reported in one particular case that the drop in run time was from 1:30 to 18 seconds, 5 times faster.

Prequisites

You should have a *nix system as a MySQL Sandbox (see below) works only there. OSX probably will do, too. This system should have some bash-like shell (obviously) and Perl installed. Kernel should have tmpfs support. Your nonprivileged user should be able to mount filesystems, or you'll need to hack the script to introduce sudo at mount step (assuming your user can sudo).

For isolated MySQL instance you need the MySQL distirbutive downloaded from the website. For isolated MongoDB instance you need the MongoDB distirbutive downloaded from the website. Note, however, that the whole MongoDB server is contained in just a single binary file ~8MB in size.

Of course as we will work completely in memory you have to be sure that you have enough RAM to store your (presumably test) datasets.

Ramdisk

Making ramdisk is very simple with latest Linux:

mount -t tmpfs $RAMDISK_NAME $RAMDISK_DIR

as root.

RAMDISK_NAME is some identifier for mountpoints table. RAMDISK_DIR is the directory which will be turned into RAM-based filesystem.

After this mount action, anything you put into RAMDISK_DIR will be placed into memory, without interaction with the physical hard drive.

Of course, it means that after unmounting the ramdisk everything which was in it will be lost.

Shutting down the ramdisk

Just unmount the created mount point:

umount $RAMDISK_NAME

as root.

Note that you probably should stop all running services which still use the ramdisk prior to unmounting!

Isolated MySQL instance

We'll use the MySQL Sandbox project to launch isolated MySQL instances.

For it to work you need the MySQL distirbutive downloaded from the website.

MySQL Sandbox is installed with the following command:

# cpan MySQL::Sandbox

as root, and you'll need to run it as follows:

SANDBOX_HOME="$RAMDISK_DIR" make_sandbox "$MYSQL_PACKAGE" -- \
  --sandbox_port="$MYSQL_PORT_DESIRED" --sandbox_directory="$MYSQL_DIRNAME"

It's a one-liner split to two lines for readability.

Note that you need root privileges only to install the MySQL Sandbox application itself, all further communication with it will be done from unprivileged account, most possibly the same under which you launch the test suite.

We need to set the SANDBOX_HOME variable prior to launching the sandbox factory because that's how we control where it'll put the sandboxed MySQL instance. By default it'll use $HOME/sandboxes, which is probably not what you need. Note that RAMDISK_DIR is the same directory that the one we prepared in previous step.

MYSQL_PACKAGE is a full path to the MySQL distirbutive package downloaded from website. Please note that MySQL Sandbox will unpack it to the same directory and will essentially use this unpacked contents to launch the sandboxed MySQL. So, probably, you'll need to move the package to ramdisk, too, to increase performance of actually launching and running the MySQL server itself, however, note that unpacked 5.6.0 contents are 1GB in size.

Remember the MYSQL_PORT_DESIRED value you use here, because you'll need to use it to configure your test suite to point at correct MySQL instance.

MYSQL_DIRNAME is of least importance here, because it's just a name of a subfolder under the SANDBOX_HOME in which this particular sandbox will be put.

After make_sandbox ended it's routine you can check that your sandbox is indeed working by running:

"$RAMDISK_DIR/$MYSQL_DIRNAME/use"

Connection to Isolated MySQL Instance

You should use the following credentials to connect to sandboxed MySQL:

* host     : '127.0.0.1'
* port     : $MYSQL_PORT_DESIRED
* username : 'msandbox'
* password : 'msandbox'

Please note that you must use 127.0.0.1 value for host and not a localhost as usual, because of sandbox internal security configuration.

Shutting Down the Isolated MySQL Instance

To shutdown the sandboxed MySQL, issue the following command:

"$RAMDISK_DIR/$MYSQL_DIRNAME/stop"

or more forceful

"$RAMDISK_DIR/$MYSQL_DIRNAME/send_kill"

This commands are needed mostly to stop the working daemon; after the unmounting of ramdisk all of sandbox data will be purged out of existence.

Isolated MongoDB instance

MongoDB server is contained in just a single binary file so it'll be a lot more easier compared to MySQL.

You'll need the MongoDB distirbutive downloaded from the website, too. This time unpack it to some directory.

After that, you can launch a separate instance of MongoDB with the following command:

"$MONGODB_BIN" --dbpath="$MONGODB_DIR" \
  --pidfilepath="$MONGODB_DIR/mongodb.pid \
  --port $MONGODB_PORT_DESIRED \
  --fork --logpath="$MONGODB_DIR/mongodb.log"

MONGODB_BIN is a /bin/mongod path preceded by the full path to the unpacked MongoDB distributive. Here you can even use your system MongoDB package, in case you have it installed. As a full example, MONGODB_BIN can have a value of ~/systems/mongodb-linux-x86_64-2.4.4/bin/mongod

MONGODB_DIR is a path to directory under RAMDISK_DIR to which this MongoDB instance should put it's files. For example, it can be just a $RAMDISK_DIR/mongo.

As with MySQL, MONGODB_PORT_DESIRED is a crucial parameter to specify the correct MongoDB instance to connect to. Remember it as you will need to set it up in your test suite.

Connecting to Isolated MongoDB Instance

By default MongoDB do not enforce any usernames or passwords so you need to just use the hostname and port parameters.

* host : 'localhost'
* port : $MONGODB_PORT_DESIRED

For example, for PHP Mongo extension, you get a connection to this instance as follows:

$connection = new MongoClient("mongodb://localhost:$MONGODB_PORT_DESIRED");

Shutting Down the Isolated MongoDB Instance

As you provided the --pidfilepath commandline argument when launching the MongoDB server, the following command should do the trick:

cat "$MONGODB_DIR/mongodb.pid" | xargs kill ; rm "$MONGODB_DIR/mongob.pid"

Essentially we are feeding the kill command with the contents of pidfile and removing it afterwards.

Bash scripts to automate the sandbox starting and stopping

There is a GitHub repository with the example scripts nicely laid out along with the comments.

There are three scripts:

  • db_sandbox_properties.sh: this is the all variable parameters you need to properly setup the sandboxes.
  • db_sandbox_start.sh: this script you run before your test suite.

    It was updated with the command to copy the tables schema from source MySQL instance to sandbox MySQL instance, so, if you have an accessible up-to-date MySQL instance with the schema for your tests, this script will copy schema to sandbox so you will have a DB ready to accept test cases.

    NOTE that you will need the sudo rights for your unprivileged user to successfully mount the ramdisk. If you do not have them, you can hack the mount command in any way you see sufficient to successfully mount the ramdisk.

  • db_sandbox_stop.sh: this script you run when you don't need the sandbox anymore. It'll stop both MySQL and MongoDB and unmount the ramdisk (note that you'll need the sudo rights for this, too).

среда, 12 июня 2013 г.

Как запускать что-либо при запуске Debian

Допустим, хочется что-то запускать прямо при старте Debian. Вот необходимые действия для этого:

# vim /etc/init.d/myscript
# chmod +x /etc/init.d/myscript
# insserv myscript

В файле /etc/init.d/myscript должно быть по минимуму следующее:

#!/bin/sh

### BEGIN INIT INFO
# Provides:          myscript
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Run some stuff
# Description:       Some descriptions
### END INIT INFO

Подробности об этой шапке можно прочитать в вики Debian. Название скрипта myscript должно быть в точности повторено в названии файла скрипта, в поле Provides и при вызове insserv.

Ранлевелы можно посмотреть здесь: http://wiki.debian.org/RunLevel.

Про dependency-based-boot там же в вики написано.