QA Fest 2019
Conference #1 on Testing and Test Automation

Opening Ceremony

Opening Ceremony

-

-

-

Quality Assurance vs. Testing

Most of us testers are called these days as QA engineers, which sounds pretty confusing to me. Testing and Quality Assurance are two different process areas, which may not even overlap sometimes. However, due to this configuration, most testing activities may not be performed as good as they could. Let's think whether we understand QA right and whether we really have to be testers in order to assure quality.

Quality Assurance vs. Testing

Most of us testers are called these days as QA engineers, which sounds pretty confusing to me. Testing and Quality Assurance are two different process areas, which may not even overlap sometimes. However, due to this configuration, most testing activities may not be performed as good as they could. Let's think whether we understand QA right and whether we really have to be testers in order to assure quality.

-

-

-

Authentication and Authorization (AuthZ & AuthN) testing: it's not only about the login form

Testers are often asked at interviews to test a login form. And this is where their acquaintance with authentication testing ends. We'll talk about authorization and authentication (AuthZ & AuthN) testing: what is the difference between them and how to stop confusing them; what kinds of AuthZ & AuthN are on the market; what are the specifics of the work of the Oauth 2.0 and OpenID protocols; what are the best practices of AuthZ & AuthN security testing and where to practice testing of that famous login form.
The talk will be useful for functional testers and those who are interested in the technological aspects of AuthZ & AuthN.

Testing in DevOps practices. Do you have a plan, Mister X?

Very often I faced with the following question: 1) We have Jenkins. 2) OK. How do you prove that your code works? 3) We have the Jenkins and the tests. 4) OK. What kind of tests do you have? When are you running them? What the plan of test running? 5) We have no plan. We have Jenkins and tests only. The theme of presentations the testing in the DevOps practices. We define what kind of tests we have. Why the tests can be useless and how we can help engineers create a good test plan.

How find vulnerability in mobile application

Every year, mobile applications are becoming more and more, but few people pay attention to the security of this application when it is under development. Since the business aims only to tear off most of the users who will use this application, they pay attention to the privacy of their customers last. In my report, I will tell you how the manual QA can test a mobile application for vulnerabilities and find the top holes rated by OWASP. The presentation will use these tools Santoku Linux + Genymotion.

Schrödinger`s code or why and how we test our product

The development of modern software solutions requires the maintenance of an effective testing system consisting of a large number of components and setting the requirements for all stages of the software development process.
Vladimir Nikonov, head of development at Terrasoft, an expert in application design with 10+ years of experience, will share his expert opinion with the QA Fest attendees to be armed with:
- the tools and process at each stage of the creation and delivery of functionality: from unit-tests to non-functional testing;
- the requirements for testing tools and competencies of QA-engineers team who needs to be advanced at each testing stage;
- the way how to introduce modern approaches in the existing project with minimum costs;
- the best practices of team development and the testing process as a whole.

Test to code mapping, code coverage by automated and manual tests, actionable coverage – easy! We made it, open sourced and free!

Have you ever collected code coverage with tests? Most likely it was unit tests. But have you tried to collect them from automated tests, manual tests? What if you even had an incredible number of 100% coverage, does it give you any understanding? Can you make it actionable? In order to solve this issue - we wrote our own tooling, made it simple and pluggable, without app code affect, and accidentally did more than competitors, but open-sourced. With this talk, I will introduce the new tool, share our approach, describe benefits of test to code mapping, tell you how to make coverage numbers actionable, introduce first steps to try it at home, and will share our plans going forward and how we plan to shift black-box testing to white-box.

A/B test for 250 million people

Indeed is the #1 job site in the world(1) with over 250 million unique visitors(2) every month (1 - comScore Total Visits, March 2018, 2 - Google Analytics, Unique Visitors, September 2018). At Indeed, our mission is to help people get jobs.
We strive to test every new feature and every improvement in every product at Indeed, and we measure the impact of those changes to ensure they are helping us achieve our mission. It is possible only with the usage of A/B tests.
A / B testing is a powerful tool for improving the performance of your product. We will talk about simple and complex questions:
- Why do we need A / B tests?
- How to conduct tests and what framework is used at Indeed.com?
- How to test tests and shared functionality?
- How to analyze the results?

Mastering the Art of Reading Code

Reading source code is usually underestimated by many developers and AQA engineers as they are mostly focused on writing it.
Moreover, some people find reading someone else’s code boring, frustrating or even creepy. Others consider this activity a waste of time. However, this is a common misconception which must be cleared. Do you want to know why? Are you still wondering how to master the art of reading code? Then come and hear it! This talk will be helpful for newbies who got stuck and don’t know how to start reading and understanding someone else’s code, as well as for professionals who want to structure their knowledge and learn some new patterns.

Performance testing in 5 minutes. How to and tool comparison

In the beginning, it was planned to tell ordinary “success story” on how we’ve done performance testing in “5 minutes”. But I’ve got advice to do something more and now we have performance tool comparison and some thoughts on how to use them.

To go up you need to have a back up

When it comes to promotion there is one question that often comes unexpected -- If you get it, who would take over your position? While it's relatively easy to onboard regular team members - preparing a successor is always a challenge as you need to find a proper balance between time, responsibilities and opportunities. You should also ensure that the team itself would sustain the changes and become stronger in time. Let's have a look at what you can do in order to build a strong team with clear successor line and how to organize handover with no risks.

Testing AI-application

Testing is an integral part of the development process. Nowadays, testing is undeservedly deprived of attention in the process of developing applications using artificial intelligence.
The speech will try to consider the errors that can occur in applications using artificial intelligence. We will also analyze typical AI tasks and what mistakes they have. we will see what, where and when should be tested in such applications. Let's look at the stages and levels of testing for such projects. We will try to open the veil of the interpretability of the artificial intelligence and what steps should be followed so that users are satisfied, and the artificial intelligence did not hide that it passed the Turing test.
Let's start asking the right questions for ML engineers who have everything working without errors.

Testing laboratory with your own hands

At Rozum Robotics, we develop and manufacture robots, servo motors, electric motors, and a bunch of different incredible things. But, testing another “uber-like ...” or “social network for ...” is not the same as dealing with real products that are ground, cut, processed and collected by gray-haired men and women with their callused hands. That's the way of testing the story will go. A year ago, we started to create our own testing and technical quality assurance laboratory from scratch, and now we have gained considerable experience, which we want to share.
What should a QA engineer think about moving from information technology to real-world testing? - You are welcome to the presentation.

WebdriverIO + Puppeteer. Double gun - double fun

In the automation world, W3C Webdriver HTTP protocol has been successfully used for a long time. With its help, many projects and libraries in various languages have been implemented (selenide, protractor, webdriverio and thousands of others). But more and more teams decide to use Chrome Debug Protocol, in particular, the Puppeteer tool. It is based on WebSockets and has its own special features - two-way communication, ability to subscribe to events in the browser, and much more. In this talk, we will look at the capabilities of both protocols, experiment and combine them together in one project to make the browser work at full power, and take the best from both communication channels.

Developing Tools, While Testing

Our daily work is comprised of testing a product and improve its quality. However, here and there, we can come to a state where we find a need to build a tool, that can make our work easierbetter.
I will share from my experience when I found myself in a situation where building a tool was needed.
We will start with a web application that allows you to know when a food delivery you ordered arrives to the office, and then we will focus on a tool that tests the performance of an app from the UI side. We will do a live demo for both of them.

Mission /*im*/possible: layman interviews SDET

Have you ever launched a test automation team on the project? Experiencing that matchless feeling when you are about to interview technician, having no technical expertise? If yes, this presentation is for you.
Here you will learn how to analyse candidate's seniority, technical skills and ability to start a team. This opportunity is conceivable even for those, who has no hands-on experience in test automation.
Drop in, it will be fun!

Future is here or how to test natural language processing (NLP) in chatbots.

Some 3 years ago, people didn’t truly believe robots could enter our lives and simplify everyday routines. Today, artificial intelligence has partially taken over human jobs, helping businesses achieve their goals and becoming a priority area of development.
With a growing demand for chatbots, there's a constant increase in the number of tools for their development, changes in technologies and complexity of tasks to be fulfilled by a robot. By 2022, users’ conversations with chatbots will save about $ 8 billion thanks to NLP (natural language processing) system, which is a major component of chatbots. Depending on the tasks performed by the NLP, the cost of an error can be rather high. The topic will cover the theory of natural language processing as a part of a chatbot as well as the development of a testing strategy for NLP models: how to distribute priorities, which tasks can be automated and how to successfully launch chatbots in an endless race for innovation.

Applications testing without using its GUI. Or use the ELK stack to analyze of the application work through the logs

In our work as a tester, we often come across testing various applications: mobile, web, desktop, embedded systems, and so on. Each product has its own interface and color. We test, find bugs, write them out in the bug tracking system, then re-check. But how often do we look "under the hood" of our car? Yes, we move to a new level, test the API, write Unit tests, test other elements of our application. But what if you can check your product in spite of all its color, its entire interface?
This talk will describe our approach using the ELK stack, which we used on one of the projects to find bugs that elude us. Those bugs, when the interface says: "My friend, everything is fine with us." and then everything turns out wrong on the prod.

Web and Mobile testing with Selenium, JUnit 5, and Docker

Selenium has become the de facto standard framework for end-to-end web testing nowadays. It allows to drive programmatically web browsers, such as Chrome, Firefox, etc. This talk presents Selenium-Jupiter (https://github.com/bonigarcia/selenium-jupiter), an open-source JUnit 5 extension aimed to provide seamless integration with Selenium and Appium. JUnit is the most popular test frameworks for Java and one of the most influential in software engineering. JUnit 5 is the next generation of JUnit, providing a brand-new programming and extension model called Jupiter. Selenium-Jupiter is a JUnit 5 extension to use Selenium and provides a rich variety of features, including the capability to use of local or remote browsers. Moreover, it allows to use browsers and Android devices in Docker containers out of the box. Selenium-Jupiter can be used to carry out different types of test of web applications, including functional, compatibility (i.e. using different browsers types and versions) or performance (i.e. using a big number of browsers) tests.

Performance testing of distributed async systems

Usually during the load testing process, the necessary app-side metrics can be obtained directly in the load generator(response time, throughput, ..). We send a request, receive a response, and response time is exactly what we need.
But what if, after the server has given you the answer, a number of asynchronous operations occur, the execution time of which we need to check? How to measure the execution time of these queries? What part of the system is the performance bottleneck?
In the report, we will consider what type of challenges appear in such a situation and how they can be solved.

How to exit testing and do I need to exit?

What exactly in testing was useful to me for my future career, and which “patterns” did not work either in testing or in development.
Testing is not “the beginning of the path”, it is a “hub” or “terminal”, having flown you have the opportunity to choose the next waypoint.
Go out or stay in testing and how to make this decision.

How to exit testing and do I need to exit?

What exactly in testing was useful to me for my future career, and which “patterns” did not work either in testing or in development.
Testing is not “the beginning of the path”, it is a “hub” or “terminal”, having flown you have the opportunity to choose the next waypoint.
Go out or stay in testing and how to make this decision.

Localization testing is the new black!

While there is debate about future technologies and the robotization of everyone and everything, Localization testing with ever new aspects of checks when introducing products to different markets is gaining momentum!
If you still think that we are only talking about translations, I invite you to learn about a dozen localization parameters. Well, for those who are already in the subject, I saved unexpected cases and sensational stories.

10 Things I Hate About test automation

People often ask why I hate Page Objects, TestNG, ReportPortal, try/catch, loops, ifs, implicit waitings, explicit waitings, Dependency injection, Spring, etc. in tests.
I will talk briefly - 5 minutes for every topic.

Injections - 4 ways of Penetration

Everyone knows about 2 types of injections: SQL and HTML, and we will also look at XML and command line. You say this is not relevant, but 28% of sites are infected with this vulnerability and the most important thing is that with this vulnerability a hacker can change the Front-End part of the site. And with the help of command-line injection you can see the structure of the server, and indeed get access to the server.

The future and its technology. What to prepare for today

What is changing in the IT sector, in business and in social relations? The role of testing, business and social practices in the new world. How to be successful in the future.

The future and its technology. What to prepare for today

What is changing in the IT sector, in business, and in social relations? The role of testing, business and social practices in the new world. How to be successful in the future.

Afterparty. Stand-up

Afterparty. Stand-up

-

-

-

Test Automation Consultancy

Why would you like to become a "Consultant". Where does it begin? And what to do if you're selected to consult a big Bank abroad :)
Successful Test Automation and Performance Testing Consultancy case study will demonstrate the whole process from finding the client through assessment, evaluation, and implementation to a happy client accepting the project.

Self-healing test automation 2.0. The Future

We have talked already about self-healing test automation, how it works, pros and cons of different approaches and about the new tool that is developing in EPAM. Our product completes the POC stage and it’s time to share the results and understand how self-healing automation will help your tests become more stable? Or vice versa, broke your automation? ... Come and find out!

Load testing which you always wanted

Десь рік тому ми почали працювати над новою версією наших продуктів. Саме тоді ми почали випробовувати різні технології, архітектури, підходи, а головне це — міряти performance, бо без цього в highload проектах взагалі не вижити.
При проектуванні “любої” системи нам потрібно знати її ліміти:
- скільки паралельних запитів може обробити мікросервіс за допустиму latency?
- як багато запитів може витримати база даних, яку ми використовуємо?
- як довго потрібно чекати на Push повідомлення?
- як довго триває розподілена транзакція та між якими сервісами відбувається найбільша затримка?
І таких питань у нас було безліч. В процесі тестування ми використовували різний tooling: JMeter, ab, Gatling, але всі вони надавали дуже лімітовані можливості. Нам не вдавалося нормально покрити push flow (WebSockets/SSE), різні бази даних, було складно імітувати різний workloads (update/read).
На цій зустрічі я розповім про наш досвід застосування load testing:
- що використовуємо для тестування баз даних, мікросервісів;
- як готуємо Pull/Push тести та як адаптуємо тести під різні протоколи (HTTP/WebSockets/SSE);
- які виникають проблеми з замірами latency.
Моя доповідь дуже практична, тому після неї ви зможете з легкістю почати застосовувати load testing у себе на проекті.

Managment of testing processes for remote teams

There is stereotype that a remote employee is a lounger relaxing on the beach with a laptop in hand. When a team is completely distributed and works remotely, you encounter difficulties, because working approaches for office employees do not work at all for remote ones. I want to share experience, how we built the working mechanism of a remote QA team, how our processes differ from processes in teams working on-sight, what challenges we've faced and what the result we achieved.

What Ukrainians should know about their soft skills

Working with fireign customers and colleagues, studying cultural peculiarities of other nations we once set a goal for ourselves to find out how Ukrainians look like in foreigners' eyes, do they have to adjust their way of communication when they work with us, is there anything that they cannot accept at all.
During the talk we will share our findings and will also touch the following topics:
- what Ukrainians should know about their soft skills,
- what are the differences between soft skills of Ukrainians and people from other countries,
- why soft skills are important when you are communicating with foreign colleagues,
- why soft skills are important for your carreer growth.

Parallel automation test writing and other elegant ways to speed up your feature delivery

When I joined the Badoo iOS QA team, we used to focus mainly on manual testing. This led to the manual regression deadlock:
- there was no time to write tests because we mostly tested manually;
- we mostly performed manual testing because there was an insufficient automation test coverage.
Nonetheless, we were able to set up our automation system and processes, break the vicious circle mentioned above and start to write decent tests.
In my talk, I will share with you how we managed to reduce the manual regression from 90% to 30% of the working time while maintaining a good level of quality and growing professionally as a team!

Snapshot testing with native mobile frameworks

I'll tell about mobile testing, by means of a screenshot comparing, using native frameworks (Espresso & XCTest) and third-party libs. I'll reveal the secret that the screenshot comparison is available in both Unit and UI tests, I'll consider which elements and in what form are available for verification and I'll share the happiness and pain of living with snapshots.

Security testing of APIs

Security testing of the APIs has some specifics compared to security testing of the web. In my talk I will walk you through main vulnerabilities that exist in APIs and how to find them. We will also talk about the tools that you can use to automate API security testing and which tool will be more suitable for what application. The talk is aimed at audience who has basic knowledge of security testing and understands main injections, as I will not stop on detailed explanation of them.

How not to get lost in IT communication networks

Is your mom the only person who takes note of your genius? Do your ideas and projects like your cat only? Your fellow students are department managers, and you get stuck between middle and senior. It’s time to search for bugs not only in a project but in your head too! Level up communicative skills!

Turn automation education upside-down

Courses, courses are everywhere. A lot of people attend testing and automation courses either offline or online before getting a job. Part of us - attending such courses even during work. But the main issue that we face right now - it that effectiveness of the courses are really low: knowledge is getting forgotten without proper and thorough practice. In my talk, I am going to share my experience in providing internal automation courses for QA Engineers using an interesting educational concept - "flipped classroom". We will take a look at how to get rid of boring theory lessons and get as much practice as possible out of education.

Testing Big Data solutions fast and furiously

We all know for sure how to test REST APIs with N endpoints, even with relational and NonSQL DBs. The same about testing UI: Selenium, Selenide, Selenoid, Protractor - all these buzzwords are well-known for us. We are also familiar with building robust, extensible and really cool test frameworks for such apps. But what about Big Data solutions, that have no back-end and no front-end parts. How to test them? What are the most critical parts? And, as usually, there is a need to introduce automation for such projects, and make it in a proper and extra-effective way.
I will show you how to deal with this and how to build test automation for Cloud Big Data solutions from scratch. As well as build it in an up-to-date and most suitable way using the most interesting libs and approaches.

Bulletproof Selenium Cluster

Browser tests are known to be the flakiest ones. This is partly because browser infrastructure is complicated to maintain. But the second reason is – main stream browser automation tools such as Selenium server are far from being efficient.
A year ago I have shown Selenoid - a truly efficient replacement of the standard Selenium server. This year I would like to demostrate how to organize a fault-tolerant and easily scalable Selenium cluster using virtual machines in the cloud. I will start by setting up several Selenoid nodes and configure them to send logs and recorded videos to S3-compatible storage. Then I will run multiple Ggr load balancer instances allowing to use all running Selenoid nodes and organize a single entry point to the cluster. Finally we'll discuss how to work with VNC and video recording in such cluster.

The earlier - the more profitable, but never too late. Why does software security begin with testing?

The talk is dedicated to the economic and cultural constraints of application security. When it comes to the security of their products, software engineers share two fundamental beliefs:
1. If “done right”, the software will be secure. 2. They can “do it right”.
Very often none of these are true. Crippled by these misconceptions, software engineers rush into the construction process and miss an opportunity to apply basic security principles at the beginning of the development process. As a result, they often find themselves dealing with tens of critical flaws once an experienced security tester gets ahold of their final builds. Fixing bugs at the end of the software development lifecycle proves to be hard, expensive, and stressful.
As implementing a full-scale software assurance program is often an overambitious endeavor, the author recommends implementing a basic subset of security practices early in the development process. And security testing is, of course, a strongly recommended first step.

Between Motivation and Burnout

Let's talk about motivation in simple terms, and clarify what motivates us to work better. Let us examine the reverse side of motivation - burnout. We will find out how to diagnose burnout and prevent unpleasant consequences.

“Hi, do you have 10 minutes?, we need to urgently talk” or about how people leave the company

The report is a kind of reverse side of the medal, as opposed to a large number of reports and articles of the type “Challenge of the Young Team Lead”. They usually talk about recruiting people, setting goals, taking responsibility. We will talk about the sad (but not always) moment when an employee decides to leave the company. How to act as a manager when an employee tells him that he is leaving? Is it possible to keep him, and if so - how? How is the fate of the counter-offer? What questions should not forget to solve in the last week? Why it is not always sad (but often).

Medical software testing — challenges and opportunities

Medical device design and manufacturing is a regulated business. Government agencies across the world enforce medical device safety and effectiveness. Neglecting regulatory compliance poses a threat to human life and health. How does regulation affect the way of working in a manufacturing company? We will discuss challenges and opportunities a medical software tester meets.

Bulletproof Selenium Cluster

Browser tests are known to be the flakiest ones. This is partly because browser infrastructure is complicated to maintain. But the second reason is – main stream browser automation tools such as Selenium server are far from being efficient.
A year ago I have shown Selenoid - a truly efficient replacement of the standard Selenium server. This year I would like to demostrate how to organize a fault-tolerant and easily scalable Selenium cluster using virtual machines in the cloud. I will start by setting up several Selenoid nodes and configure them to send logs and recorded videos to S3-compatible storage. Then I will run multiple Ggr load balancer instances allowing to use all running Selenoid nodes and organize a single entry point to the cluster. Finally we'll discuss how to work with VNC and video recording in such cluster.

It's just too Slow: Testing Mobile application performance

Mobile apps and websites are now the predominant ways that users interact with brands. Research has shown that slow sites and apps lose customer engagement. Despite this, most mobile sites and apps have performance issues that can be easily resolved once diagnosed. In this talk, we will walk through steps to diagnose network performance bottlenecks in mobile services. We'll discuss real world examples and how they were resolved. Attendees will leave this talk armed with the tools to test, diagnose and resolve the top network performance issues that affect mobile today.

Tune your P’s: the pop-art of keeping testability under control

Testability is being discussed pretty often lately. More often it is limited to ability to test or rather to gain certain automation coverage. However, 10 P’s of testability technique exists and can be used to analyze, tune and improve development process in general.
This is exactly what we going to discuss.

How to earn more

Do you know the term mindshift? This is what you will experience after this talk. It will not be about QA processes or tools, it will be about money and business, about risks and communications. All this with examples from Ukrainian and world IT in the format of live communication with the audience.

Efficient quality evaluation

This talk is about all-inclusive quality evaluation.
About proven management practices (Evidence Based Management, Management 3.0) and their fit to Quality Assurance domain. About process management background and metrics must-haves like attributes, visualization and balance. Also, I will demonstrate QA Metrics Ecosystem developed based on M3.0 Scoreboard Index approach.

Mocks and network tricks in UI automation

Web applications and technologies are developing rapidly. We have already entered the era of Single Page Application and are heading towards the Progressive Web Application. In most modern projects, the teams are divided into front-end and back-end, and not only teams, but there is a separate release policy. This requires more detailed front-end testing approaches. In this report, we will look at cases that we have in practice when testing front-end tasks and automation tools that can solve the tasks described in these cases: reading request / response browser network and correspondingly stubing responses.

Web security in 40 minutes

Let's talk about the most common mistakes that web application developers assume, and how an attacker can use them in their favor. We capture the maximum amount of material in a short period of time.

QA 3.0. New generation

In parallel with the development of IT industry, the profession of tester evolved very much - from "monkey testing" responsibilities that reduce developers' load, to a profession that is capable of earning more than these developers. Quality assurance at these days is not just about regressing or even automating it. It is an improvement on all stages of product development and new generation of testers are really capable of doing it.
We will discuss this path of evolution, review all their components and causes. I will tell how you can build a QA culture in an organization and develop testers 3.0.

QA Hackathon - The Cookbook

Would you like to see more QA events in Ukraine? Feel that there’s a lack of ones? Guess who can change the situation?You can!
I’m gonna share the approaches we have used to organize QA Hackathons at Wix so you can build your own great hackathon.

N/A

Selenide for advanced users

If you are tired of primitive "Hello World" examples and want to know more about internals of Selenide and typical problems, you are welcome!
We are going to discuss parallelization, rules and listeners, and tricks with JavaScript and proxy.
You can even suggest your own topics. Influence the content!

Breaking into information security

Why everything is spoiled

In this talk, I will cover the pain points of the Test Automation process. We will discuss traps, mistakes and crazy decisions that lead to test automation failure and lost budgets.

Why everything is spoiled

In this talk, I will cover the pain points of the Test Automation process. We will discuss traps, mistakes and crazy decisions that lead to test automation failure and lost budgets.

Closing Word

Closing Word

-

-

-