CUSTOMER BACKGROUND
Horoshop is a Ukrainian e-commerce company that provides a namesake SaaS platform for running online businesses. Horoshop has over 3500 customers; the platform hosts around 8 thousand e-shops and nearly 18 thousand domains.
The company’s web toolkit allows for creating fully-fledged, easily customizable e-shops from predefined design patterns, and includes marketing tools for analytics, SEO, and integration with 3rd party services. An advanced set of features and flexibility of use enable Horoshop to successfully compete with marketplaces and other e-commerce solutions such as Shopify, WooCommerce, and Opencart.
WHY THEY CHOSE US
When asked about the deciding factors for working with SHALB, Igor Stolnitsky, CEO of Horoshop, cited technical expertise and a deep understanding of customer needs demonstrated by our team:
The specialists at Shalb did not just fulfill our task. They studied our business and understood our needs better than we did. And they offered their proposal on how best to solve our problem.
When our collaboration started, Horoshop was on its way to becoming a fully-fledged SaaS platform. SHALB development work helped the company reach new heights, providing it with a reliable server architecture to support its business initiative, and a CMS to control customers on the platform.
PROJECT DESCRIPTION
Infrastructure
Horoshop infrastructure runs on a bare-metal cluster managed by Docker Swarm. It mainly uses three types of servers to run the workloads: Swarm working nodes, MySQL database servers, and NFS storage servers (recently migrated to Amazon S3).
A pivotal element of the Horoshop infrastructure is Rundeck – a workflow orchestration system that drives most of the company’s inner processes. Basically, it is a console to manage and work with the Horoshop platform, providing for adding new customers, setting them up, or changing domain configuration. In addition, Rundeck is responsible for the CI/CD process of running Shell scripts for building images, code delivery, and version upgrades. It also accounts for the support of multiple platform versions – an additional functionality developed by SHALB that enables VIP features for the Horoshop customers.
To codify a bare-metal Horoshop infrastructure, we used Hiver – a customized tool developed by one of the leading SHALB experts, Roman Progonnyj. Hiver combines the functionality of a package manager, IaC, and CI/CD tools. It builds cluster manifests, applies templating to render them, and deploys rendered manifests to Docker Swarm. As an IaC tool, Hiver plays a key role in the setup of new servers.
Databases
Most of Horoshop’s applications are database-centric, which makes their performance largely dependent on the state of databases. The growing number of Horoshop customers led to an increase in database queries and write operations, causing Horoshop’s systems to underperform. It was a core problem that could not be solved by simply adding more hardware resources to a database server.
To scale Horoshop’s databases, we used sharding – a method of partitioning large databases across several servers. The approach allows for the scaling of sharded systems to a required level of capacity without adding resources to a single server. We partitioned Horoshop’s systems on a customer basis and dedicated an independent MySQL cluster for each shard. Achieving a balance between database size and system resources helped us effectively address the performance and scalability challenges of Horoshop’s systems.
Domain management
Horoshop accommodates nearly 18 thousand domains on its platform. Each domain has its own settings that define the resources to be used – databases, NFS storages, SSL certificates, etc. – and the type of content to be displayed in response to a query. With over ten thousand domains in service, it has a myriad of parameters to deal with. Horoshop was struggling to provide effective management of its customers’ domains.
Each domain was defined in Nginx as a virtual host with its own pack of settings. The domain parameters accounted for users’ redirection to particular websites and kept the customers’ data isolated. However, as the number of Horoshop customers grew, managing all domains in a single config became a nightmare: the Nginx process increasingly demanded more time and resources to complete. It was a bottleneck problem that SHALB specialists effectively solved by adding one more Nginx to the cluster configuration.
Now, one Nginx controller acts as a reverse proxy to accept HTTP and HTTPS web queries, expose them to services within the cluster, and provide for SSL termination. The other controller is configured in the application backend to perform rewrites, redirects, and checking queries for certain parameters. Distributing Nginx functionality allowed us to reduce its operational load and diminish the size of Nginx config, making it easily runnable.
Solving the issue with SSL certificates
Horoshop relies on Let’s Encrypt to enable secure connections on the platform. But how to provide certificates for all its customers without hitting specified limits and keep them up-to-date? Considering the number of Horoshop domains, it was a difficult task to accomplish. SHALB specialists developed a customized controller that automatically obtains new certificates for Horoshop customers and configures them for use. It also checks the certificates’ expiry date and updates them on time without human intervention.
PROJECT OUTCOME
On completion of the project, Horoshop acquired a reliable infrastructure that ensures the stability of its systems. The bare metal cluster run by Docker Swarm contains everything to provide Horoshop’s platform with its vital capacity, which is:
- A system for releasing new platform versions
- CMS for customer management and updating client websites
- Effective solution for domain management
- Analytics and monitoring of all server metrics and platform events by Grafana and Kibana
- Operational support 24/7.
Over five years, Horoshop’s server cluster has gone through 2 global architecture upgrades with no platform downtime. According to Igor Stolnitsky, the solution proved its quality over time: the easy and cheap to scale cluster allowed for a tenfold increase in Horoshop’s customer numbers while still keeping the size of its team unchanged.
CUSTOMER FEEDBACK
When asked about the aspects of our collaboration that he liked most, Igor Stolnitsky mentioned effective cooperation and the synergy of both teams.
Igor Stolnitsky, CEO of Horoshop
We have been working with the same specialists for all five years, and it felt very close to being one team.
SHALB is proud to be a part of the Horoshop success story. For the five years that our teams have worked hand-in-hand, this case proved to be one of the most unusual yet technically challenging SHALB projects.
SHALB specialists provide services for infrastructure development and maintenance for projects of any complexity. Our team is flexible, accommodating, and knowledgeable. Book an online meeting or contact sales@shalb.com to see it for yourself. We’d be delighted to hear from you!
