{"id":15499,"date":"2018-06-29T11:08:13","date_gmt":"2018-06-29T11:08:13","guid":{"rendered":"https:\/\/coherent-labs.com\/?p=15499"},"modified":"2018-07-02T13:02:44","modified_gmt":"2018-07-02T13:02:44","slug":"development-process-in-cl","status":"publish","type":"post","link":"https:\/\/coherent-labs.com\/posts\/development-process-in-cl\/","title":{"rendered":"Development process in Coherent Labs"},"content":{"rendered":"

\"development

At Coherent Labs, we build technologies and tools for one of the most critical audiences in the IT industry – game developers.\u00a0Nothing, but the best quality software will do for them.<\/p>\n

Software quality has many dimensions. In this blog post, I\u2019ll mostly focus on how we ensure the user-facing quality – the fact that the software must work correctly and perform adequately (should be very fast in our case). Other dimensions of quality include modifiability, testability, scalability, which in the end are reflected in the user-facing quality attributes.<\/p>\n<\/div><\/div><\/div><\/div><\/section>

<\/div>
<\/div>

Development process<\/h2>
<\/div><\/div><\/div><\/section>

To ensure the high-quality standard we strive for, we employ a mix of human\u00a0actions, tools & automation. Developing a new feature involves multiple steps until it finally lands in the hands of our users.<\/p>\n

For each feature:<\/strong><\/p>\n

\u2022 The developer(s) extracts all the requirements from the major stakeholders. In our case, this can include the Product Manager, Software Architect, CTO, Team lead & Design team.<\/p>\n

\u2022 The developer creates a Design document – this is a high-level document that should answer important questions such as:<\/p>\n

– What should I do?<\/p>\n

– How am I going to do it (technically)?<\/p>\n

– What risks do I see?<\/p>\n

– How will I test the new feature?<\/p>\n

– How much time I believe it\u2019ll take?<\/p>\n

\u2022 The design document is discussed with anyone who can contribute to the project. In most cases, these people are the stakeholders and developers. Design docs allow us to significantly reduce the risk for the development of new features and make sure that all members of the team are on the same page. It also has the benefit of remaining as a long-term artifact to remind us how a certain feature was designed at the beginning and what decisions were made. It can be invaluable down the road if the feature has to be modified somehow or faults are found and the developer who created it can\u2019t remember all the details. In my experience, I keep certain decisions in my head for max 1-3 weeks, after that I either have to rewind everything in my head (long and tough process) or look at some notes.<\/p>\n

\u2022 After the design is approved by the teams, the developer writes the actual code. Now he\/she knows exactly what needs to be done and how. The actual code-writing is expected to be easy and frictionless.<\/p>\n

\u2022 When the feature is ready, the developer creates multiple tests – both for the current feature and automated regression tests. The automation infrastructure is explained later in the post.<\/p>\n

\u2022 When all tests pass, the developer moves the feature in the Code review stage. Each new piece of code has to get 2 \u201cShip It\u201d from other devs. We are very democratic at this stage, every developer can review whatever he\/she wants. It usually happens that the featured creator himself appoints to the review at least 1 team member that is familiar with the sub-system. The Code review stage is important, not only because it improves the quality of the features, but also gives much better visibility to all team members. We use Reviewboard<\/a>.<\/p>\n

\u2022 After the feature has received the needed \u201cShip It\u201d, it is shown to QA. This allows the QA team to know what to expect, how to test and make sure the feature is aligned with the requirements.<\/p>\n

\u2022 At this stage, the feature is merged back into the main branch. QA works always off the main branch.<\/p>\n

The following steps illustrate the most important \u201chuman\u201d part of the process. To facilitate and make the development faster, we have introduced numerous tolls in the pipeline.<\/p>\n

 <\/p>\n

\"\"<\/p>\n<\/div><\/div><\/div><\/div><\/section>

<\/div><\/div><\/section>
<\/div>

Automation tools\n<\/h2>
<\/div><\/div><\/div><\/section>
Build automation\n<\/h5>

Our products currently support at least a dozen platforms and game engines. This means that it\u2019s unfeasible to require every developer to build & run his\/her code for each one of them. Build machines are employed that continuously compile & test the projects across multiple platforms. The coordination is performed with BuildBot<\/a> and hand-written scripts. Developers can leverage the infrastructure by running builds and test on feature branches.<\/p>\n

Build machines run 3 distinct types of builds for the products:<\/p>\n

\u2022 Smoke builds – run after every commit and make sure the product builds and passes the most important tests. Smokes have to be fast, so they only run the most basic sanity checks.<\/p>\n

\u2022 Nightly builds – run every night and cover all platforms and tests, including performance regression checks.<\/p>\n

\u2022 Release builds – when we tag a version for release, it is essentially the same as the nightly, but gets a release version.<\/p>\n<\/div><\/div><\/div><\/div><\/section>

<\/div>
Test automation\n<\/h5>
<\/div>

We run different types of test that cover the functionality of our products:<\/p>\n

\u2022 Unit tests – multiple internal systems are covered with unit tests to improve their reliability. We use Google Test as a framework for our tests.<\/p>\n

\u2022 Layout tests – these are more high-level tests that run in an application that internally uses Hummingbird\/GT. The tests usually produce a screenshot that is compared to a committed golden master. Many tests also support JavaScript logic through the mocha\/chai frameworks. They allow us to do more precise checks on specific values. For instance, a certain animation should be at a certain point after 0.5s etc.<\/p>\n

\u2022 Graphics tests – Our rendering library Renoir is extensively tested as a standalone package. It powers all our products and it\u2019s important to make sure it always works reliably in isolation.<\/p>\n

\u2022 Performance tests – we have a custom tool that runs a test suite on game consoles and checks the performance. If any profiled function on any test has become slower, the build fails. This is an extremely important aspect because seemingly subtle changes can have a negative impact on the performance of the software. The tool makes sure we always improve things. Historic results are kept in a database and allow us to plot changes over longer periods of time.<\/p>\n<\/div><\/div><\/div><\/div><\/section>

<\/div>
Additional tooling\n<\/h5>
<\/div>

Along with tests, we have an additional set of tools that are currently run by hand by QA or developers.<\/p>\n

\u2022 Code coverage – a tool is used that calculates C++ code coverage of tests. When a developer writes new features and tests for it, she can check that she has covered all the new code. QA also routinely runs coverage of all tests and looks for portions of the source that are not exercised.<\/p>\n

\u2022 Fuzzing – we have a tool that generates fuzzy tests continuously and runs our products on them. In our case, the fuzzer generates random HTML pages. The tool then runs Hummingbird on the random page and monitors it for asserts\/crashes. If such is detected, the HTML page, log file, and a complete memory dump are recorded for inspection by a developer. The fuzzer has been very successful in finding some defects that are very difficult to encode in a test by hand – for instance very large and somewhat chaotic mutations of the DOM. We use the same tool to also generate and test our declarative data binding system.<\/p>\n

\u2022 Static analysis – we periodically run static code analysis tools on the software. So far we\u2019ve had some minor bugs detected by them, but nothing really serious. I believe that is due to the strict development process we enforce.<\/p>\n<\/div><\/div><\/div><\/div><\/section>

<\/div>

Future developments\n<\/h2>
<\/div>

The next iteration of our internal tools will see better integration of the code coverage and fuzzing tool. So far, the fuzz tests are completely random – this is not ideal, because they tend to exercise the same code paths very often. This means that catching a bug becomes quite unlikely. We plan to try some genetic programming techniques to make the fuzzer smarter and guide it through the code coverage tool. Genetic algorithms have been shown to dramatically improve the effectiveness of fuzzing and we are eager to try it for ourselves.<\/p>\n<\/div><\/div><\/div><\/div><\/section>\n","protected":false},"excerpt":{"rendered":"At Coherent Labs, we build technologies and tools for one of the most critical audiences in the IT industry - game developers.\u00a0Nothing, but the best quality software will do for them. Software quality has many dimensions. In this blog post, I\u2019ll mostly focus on how we ensure the user-facing quality - the fact that the [...]","protected":false},"author":18,"featured_media":15547,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[1,410],"tags":[440,445,443,442,441,444,438,439,436,437],"yoast_head":"\nDevelopment process in Coherent Labs - Coherent Labs<\/title>\n<meta name=\"description\" content=\"The high quality of our software is our most important priority. Here's how the process of developing the best UI middleware on the market looks like.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/coherent-labs.com\/posts\/development-process-in-cl\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Development process in Coherent Labs - Coherent Labs\" \/>\n<meta property=\"og:description\" content=\"The high quality of our software is our most important priority. Here's how the process of developing the best UI middleware on the market looks like.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/coherent-labs.com\/posts\/development-process-in-cl\/\" \/>\n<meta property=\"og:site_name\" content=\"Coherent Labs\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/CoherentLabs\/\" \/>\n<meta property=\"article:published_time\" content=\"2018-06-29T11:08:13+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-07-02T13:02:44+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/coherent-labs.com\/posts\/wp-content\/uploads\/2018\/06\/Artboard-2-3.png\" \/>\n\t<meta property=\"og:image:width\" content=\"857\" \/>\n\t<meta property=\"og:image:height\" content=\"216\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Alexandra\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@CoherentLabs\" \/>\n<meta name=\"twitter:site\" content=\"@CoherentLabs\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Alexandra\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/coherent-labs.com\/posts\/development-process-in-cl\/\",\"url\":\"https:\/\/coherent-labs.com\/posts\/development-process-in-cl\/\",\"name\":\"Development process in Coherent Labs - Coherent Labs\",\"isPartOf\":{\"@id\":\"https:\/\/coherent-labs.com\/posts\/#website\"},\"datePublished\":\"2018-06-29T11:08:13+00:00\",\"dateModified\":\"2018-07-02T13:02:44+00:00\",\"author\":{\"@id\":\"https:\/\/coherent-labs.com\/posts\/#\/schema\/person\/b0252689790c3d1c9036209533d03791\"},\"description\":\"The high quality of our software is our most important priority. Here's how the process of developing the best UI middleware on the market looks like.\",\"breadcrumb\":{\"@id\":\"https:\/\/coherent-labs.com\/posts\/development-process-in-cl\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/coherent-labs.com\/posts\/development-process-in-cl\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/coherent-labs.com\/posts\/development-process-in-cl\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/coherent-labs.com\/posts\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Development process in Coherent Labs\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/coherent-labs.com\/posts\/#website\",\"url\":\"https:\/\/coherent-labs.com\/posts\/\",\"name\":\"Coherent Labs\",\"description\":\"Cross-Platform Game UI Middleware\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/coherent-labs.com\/posts\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/coherent-labs.com\/posts\/#\/schema\/person\/b0252689790c3d1c9036209533d03791\",\"name\":\"Alexandra\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/coherent-labs.com\/posts\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/687476ba742eac5c6da76b305bc4004b?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/687476ba742eac5c6da76b305bc4004b?s=96&d=mm&r=g\",\"caption\":\"Alexandra\"},\"url\":\"https:\/\/coherent-labs.com\/posts\/author\/alexandra\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Development process in Coherent Labs - Coherent Labs","description":"The high quality of our software is our most important priority. Here's how the process of developing the best UI middleware on the market looks like.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/coherent-labs.com\/posts\/development-process-in-cl\/","og_locale":"en_US","og_type":"article","og_title":"Development process in Coherent Labs - Coherent Labs","og_description":"The high quality of our software is our most important priority. Here's how the process of developing the best UI middleware on the market looks like.","og_url":"https:\/\/coherent-labs.com\/posts\/development-process-in-cl\/","og_site_name":"Coherent Labs","article_publisher":"https:\/\/www.facebook.com\/CoherentLabs\/","article_published_time":"2018-06-29T11:08:13+00:00","article_modified_time":"2018-07-02T13:02:44+00:00","og_image":[{"width":"857","height":"216","url":"https:\/\/coherent-labs.com\/posts\/wp-content\/uploads\/2018\/06\/Artboard-2-3.png","type":"image\/png"}],"author":"Alexandra","twitter_card":"summary_large_image","twitter_creator":"@CoherentLabs","twitter_site":"@CoherentLabs","twitter_misc":{"Written by":"Alexandra","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/coherent-labs.com\/posts\/development-process-in-cl\/","url":"https:\/\/coherent-labs.com\/posts\/development-process-in-cl\/","name":"Development process in Coherent Labs - Coherent Labs","isPartOf":{"@id":"https:\/\/coherent-labs.com\/posts\/#website"},"datePublished":"2018-06-29T11:08:13+00:00","dateModified":"2018-07-02T13:02:44+00:00","author":{"@id":"https:\/\/coherent-labs.com\/posts\/#\/schema\/person\/b0252689790c3d1c9036209533d03791"},"description":"The high quality of our software is our most important priority. Here's how the process of developing the best UI middleware on the market looks like.","breadcrumb":{"@id":"https:\/\/coherent-labs.com\/posts\/development-process-in-cl\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/coherent-labs.com\/posts\/development-process-in-cl\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/coherent-labs.com\/posts\/development-process-in-cl\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/coherent-labs.com\/posts\/"},{"@type":"ListItem","position":2,"name":"Development process in Coherent Labs"}]},{"@type":"WebSite","@id":"https:\/\/coherent-labs.com\/posts\/#website","url":"https:\/\/coherent-labs.com\/posts\/","name":"Coherent Labs","description":"Cross-Platform Game UI Middleware","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/coherent-labs.com\/posts\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/coherent-labs.com\/posts\/#\/schema\/person\/b0252689790c3d1c9036209533d03791","name":"Alexandra","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/coherent-labs.com\/posts\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/687476ba742eac5c6da76b305bc4004b?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/687476ba742eac5c6da76b305bc4004b?s=96&d=mm&r=g","caption":"Alexandra"},"url":"https:\/\/coherent-labs.com\/posts\/author\/alexandra\/"}]}},"_links":{"self":[{"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/posts\/15499"}],"collection":[{"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/users\/18"}],"replies":[{"embeddable":true,"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/comments?post=15499"}],"version-history":[{"count":34,"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/posts\/15499\/revisions"}],"predecessor-version":[{"id":15552,"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/posts\/15499\/revisions\/15552"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/media\/15547"}],"wp:attachment":[{"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/media?parent=15499"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/categories?post=15499"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/tags?post=15499"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}