Introduction

Architecture design is not only the first step in building an application but perhaps the most vital one as well. Earlier, the entire application was built as a single, indivisible unit of code that shared resources – known as Monolithic Architecture.

However, modern applications tend to favor the loosely coupled or decoupled Microservices Architecture. It allows the services to be autonomous, agile, rapidly scalable, and independently deployable. 

Another increasingly favored architecture is Serverless Architecture because it does not need server management. The servers used to build applications are managed and provisioned by Cloud Service Providers (CSPs) and cost as per consumption. 

The options available to build your application are numerous. Before starting to build your products, as a startup founder, you must understand the architecture requirements. You need to answer questions such as; which architecture is best suited for your unique solution; what benefits are offered by each available option; and among those benefits, which cater best to your priorities. The blog appeases the architecture for application development debate for startup founders. 

What is microservices architecture? What are its benefits?

In a microservices architecture, a single large application is broken into smaller elements where each element delivers a specific value. In monolithic applications, all functions are connected to a single large codebase which is harder to manage. For modern applications, the preferred application development architecture is a microservices architecture, in which the codebase is decoupled into smaller functional units that interact with each other through well-defined APIs. 

How does microservice architecture impact an application’s performance and scalability?

The biggest benefit of a microservices architecture is agility. The decoupled services are programmed, tested, and even deployed independently. Microservices are safer because the blast radius has been reduced with decoupling. Other merits are the use of well-defined interfaces for communication among the decoupled services. The backend code paths are well defined as well. Microservices are also easier to test due to the small-sized code for each decoupled service. 

In comparison, the demerit of monolithic architecture is that all developers are contributing to the same codebase. Hence, over time, the codebase increases in size and becomes harder to manage. Each team member creates their own changes. For these changes to be incorporated into the codebase, the approval requests for a large number of changes get queued. And so, the development is simply not fast enough in monoliths. 

A well-architected monolith application can outperform microservices but only in the short term. Over time, the monolithic application becomes bulkier but fragile, harder to maintain and operate. However, irrespective of the period, the agility and safety merits of microservices architecture trump the performance benefits of monolithic architecture. 

What is the difference between Monolithic, Service-Oriented Architecture (SOA), and Micro-services Architecture?

All functions whether it is the database or the API layer(s), are part of the same codebase in monolithic architecture and are maintained as one repository of code as well. Generally, they get packaged as a single large container or VM that gets deployed. Contrary to monolithic architecture philosophy is the microservices architecture which has been explained earlier. 

Microservices architecture uses the principles of service-oriented architecture. The service-oriented architecture combines discrete business functions that have been separately deployed and maintained. Its function is to provide data and code integration services and to ensure communication between these business functions. 

The difference between the two is the scope of operations and their methods of communication. Microservices architecture operates within the mobile application development services realm whereas service-oriented architecture is an enterprise-wide phenomenon. In microservices, each decoupled element uses APIs to communicate with each other. In contrast, each element in service-oriented architecture uses an enterprise service bus, (ESB) to communicate between discrete business functions.  

What are the best practices for designing microservices?

When microservices are designed, there are multiple ways to divide the application into the smallest element possible. The division can be based on teams called Service Per Teams e.g. front-end and back-end APIs teams, etc. It can also be based on functional areas that vary on the business use cases or application capabilities. 

Can breaking an application w.r.t number of transactions, serve as a guiding principle? 

Dividing application development architecture in terms of transactions can serve as a good guiding principle because the frequency of transactions per second can affect testing, deploying, and monitoring. Since there is no rule for the division parameter, the number of transactions can be one parameter. To cover all the use cases, the development team can conceive an agile way to implement, test, and monitor the application. The aim here is to achieve the best safety and performance for your application and its deployment. 

Is microservice architecture better than others in terms of cost?

It depends on the use case. Any architecture can be costly if used carelessly. For example, it is more costly if the monolith application is running in big instances or if load balancers are not properly configured. Similarly, microservices can be costly if multiple VMs or containers are in use but not needed. In general monolithic architecture tends to be slightly less expensive. 

Would you say that with a monolithic architecture, CI/CD pipelines are easier to manage?

No. It is rather hard to manage CI/CD pipelines in a monolithic architecture. One small change in a large codebase will impact other parts of the application. Whereas in microservices, the well-defined interfaces ensure fast defect isolation and resolution. 

CI/CD pipelines and monolithic architecture are like oil and water. Monoliths are harder to build, harder to deploy, and harder to test whereas Microservices are much easier in comparison. 

How do I maintain the security of my application in microservices architecture?

In terms of security, microservices and monolithic architecture are not fundamentally different because in both cases you have to perform your own security reviews.  The responsibility of patching your hosting and maintaining availability is on you. However, in serverless architecture, these actions are the responsibility of CSPs. 

Does that mean for security serverless architecture is better?

No. Serverless takes away that responsibility and hence a pain point. There is a fleet of VMs and servers that do not need management in serverless architecture, depending on the scale of your operations. However, serverless is not easy to manage when it comes to security unless the serverless functions are divided into the smallest possible functions.  Fine-grained IAM policies, then, manage each microservices that communicate with each other. This allows each team to own their own space and to autonomously work on that service. 

What are the challenges you face with serverless architecture? Please go into details of runtime and resource isolation.

One of the challenges is the time budget. Serverless can only run for a certain period. It is both a blessing and a curse. A blessing because it ensures developers finish early or at least timely. And a curse because you must finish work on serverless architecture in the designated time. This poses even more strain in high-load situations. 

Resource isolation is another challenge with serverless architecture. Serverless is no different when it comes to shared resources on the same infrastructure. Hence, the noisy neighbor effect will be a factor. The effect will be exasperated if a lot of serverless functions are running on the same infrastructure. 

How would the security, latency, and privacy of the data be impacted for a complex, distributed application in microservices architecture?

There were times when it was believed that microservices take longer to deploy. However, computing has evolved to such an extent that monolith does not yield more performance benefits than microservices, in general.

 The only time monoliths yield better performance is when IPC communication is used on the same Unix machine. Monoliths are faster in that case than using APIs. However, GraphQL query across two microservices, running on two different machines, may not be so. 

Nowadays network fabric, switching, and computing infrastructure has evolved enough to minimize the difference. 

How does microservices architecture deal with asynchronous calls resulting in congestion and latency? 

The issue of congestion and latency resulting from asynchronous calls does not change for monolithic architecture or microservices architecture. If you have a use case with high Transactions Per Second (TPS) then you design the architecture to handle high TPS. One important design consideration for monolith and microservices is to autoscale your system. Every Cloud Service Provider (CSP) allows you to autoscale. For example, for a food delivery service, the TPS will be low in the middle of the night vs during lunch hour. If you fail to autoscale, then the payment for both times will be the same. When in fact, you should pay less for times with fewer TPS. 

As a startup founder, what should be my motivation for going serverless? How would this impact my cost?

Your motivation to use serverless architecture should be agility and time efficiency. Independent teams can move faster. There is higher availability and safety of applications in a serverless architecture. Serverless also allows you to reduce your blast radius, build test cases that are specific to each microservices, and avoid unpredictable code paths. More important is the benefit of cost because you pay for only what you use. It is reasonable to run VMs as long as they are needed. All these cumulatively make serverless infrastructure an excellent choice for application architecture. Therefore, you should use serverless until you can’t. 

The cost of serverless architecture or the cost of the on-prem solution, which is easier to bear?

On-prem solutions also have serverless with VM support, nowadays. Whether you are on-prem or on the cloud, the cost will be according to the infrastructure consumption time. Serverless prevents you from hogging resources when not needed. Once the resource is released for use somewhere else, you get more from the same infrastructure. 

What are the different types of APIs that microservices can use to communicate with each other?

There are many. However, there are three standard APIs. The most commonly practiced API is the Representational State Transfer (REST). REST uses a client/server approach where the front and back-end APIs operate separately. RESTful APIs usually exchange data or documents and can communicate with each other both directly and with the help of intermediaries such as load balancers and API gateways.  

SOAP i.e. Simple Object Access Protocol is commonly used in the creation of Web API. Therefore SOAP supports a wide range of the internet protocols such as HTTP, SMTP, TCP, and more. SOAP works with XML (Extensible Markup Language). The extensible nature provides developers the ease and flexibility to add new features and functions. 

And the third is the Remote Procedural Call (RPC) usually solicits executable processes that help the exchange of parameters and their results. However, RPC is the least commonly used API because of its higher security risks and limited data-type support.

What are the best practices to implement microservices APIs?

Instead of implementing APIs yourself, use standard APIs. Make sure the standard API, you have opted for, has developer community support to back it up. There are a lot of libraries available to implement APIs. In the public cloud, even technologies resemble AWS API gateway. Other clouds have similar tools as well. They are easy to set up and can provide a restful interface that performs CRUD operations on the backend infrastructure in no time. 

Explain the way to implement service discovery in a microservices architecture.

Earlier, there might have been debates on the selection of many different service discovery architectures but currently, DNS is used by the majority. It is simple. Every service has a name. You register with a DNS and it guides the communication and service location. Service registration and service discovery are standardized on DNS and no longer require complex architectures. 

What does the future hold for these application development architectures? How do you see them evolving? 

For the application development architecture debate, winners have been called. The future of computing would be more and more serverless applications except for the workloads that cannot be serverless. For each use case, you need to choose which workload will be serverless. Therefore, it would be a mix of serverless and container or serverless container and database running as VMs. 

To Watch the Complete Episode, Please Click on This YouTube Link:

https://www.youtube.com/watch?v=9Jcse3Vjw4I&t

Introduction

Machine Learning has been around for decades. However, the term has been interchangeably used with Artificial Intelligence, Deep Learning, and many others. With machine learning increasingly becoming part of technologies from all walks of life, software development is no exception. Therefore, we deem it necessary to explicate Machine Learning (ML) and all related terms for startup founders. Centrally, the blog addresses how Machine Learning Operations (MLOps) has now become an integral part of the Software Development Lifecycle (SDLC) for many businesses.

Introduction to MLOps and Why It Matters?

Machine Learning is a set of algorithms that learn from data, unlike explicitly programmed algorithms. It is used in many fields such as spam detection, product recommendations, object recognition, fraud detection, forecast modeling, etc.

For machine learning systems, the real challenge is not to build a machine learning model; instead, the real challenge is to build an integrated machine learning system that continuously operates seamlessly in production with quick solutions to any arising issues. Practicing machine learning operations elicits the strive to automate and monitor all parts of the data pipelines such as data collection, data cleaning, feature engineering, model development, evaluation, deployment, and infrastructure management. 

Among the numerous benefits of ML, the first is the increase in development velocity. It might take time for companies to develop processes for MLOps today but it rapidly reduces the time of development for data scientists, consequently leading to the growth of the company. 

Another benefit is data and model lineage and tracking. The credibility of a prediction lies in the data source, the type and model of cleaning transformations, and what metrics are used to evaluate that model. Hence, in MLOps, every part of the pipeline is tracked and versioned, thus allowing for model audits in the future. 

Another MLOps benefit is reproducibility. In MLOps, the emphasis on version control, tracking, and modularization of all components enables developers to re-run the pipeline and produce the same results, models, and data. Upgrading a stack in MLOps is easier due to modularization and containerization of the code. These steps are taken as best practices in MLOps and lead to fewer issues in production as well.

What Are the Different Components of a Machine Learning System?

Machine learning has two main parts namely, data and algorithms. In machine learning systems, these two main parts are further divided into six sub-parts, the first of which is data collection and analysis. Based on its importance in ML systems, data is collected from multiple sources. This collected data could be structured or unstructured, therefore, it needs to be analyzed. The analysis tackles the following queries: the origin of the data, its range and scale of values, and its quality. 

The second part is feature engineering. Once the data is developed, it needs more work before it is fed into an ML model. This feature engineering work varies as per the requirements e.g. in the spam classification model, the features such as the subject line or the email body text would be developed. Similarly, for the stock market value prediction model, the feature engineering would require features such as historic prices of the stock, market indexes, market volatility, or political stability. 

The third part is model development. After the feature development, the data with highlighted features are fed into the ML model. With the evolution of ML technology, model development has become the easiest part of the pipeline. Owing to the vast and conscientious libraries, pipelines only require a few lines of code and deliver state-of-the-art ML performance. 

The fourth part is the model evaluation and validation. Once the model is built, its quality and performance for the business use-case are assessed thus providing a direction for the machine learning model and how it should be optimized. 

The fifth part is model deployment. The model at this stage is used for live predictions and a pipeline is built around it which continuously deploys and serves the requirement. 

The last part is the monitoring and it is a vital part of the ML systems. Monitoring of ML models is performed to ensure the required performance is maintained in production. It also ensures there are little to zero deviations from offline model development.

MLOps Vs DevOps: Are They Really That Different?

DevOps normally focus on application development, but MLOps is a combination of both DevOps and machine learning. The functional features of DevOps, such as CI/CD deployment, dependable releases, load testing, and monitoring, are combined with machine learning components to facilitate MLOps. 

The differences between them can be summarized as follows;

MLOps team is structured differently from a DevOps team; in that data scientists are part of the mix and they might not have software engineering knowledge. 

The second difference is the experimental development. Traditional software development is roughly linear as compared to ML pipeline which is rather circular. 

The third difference is the requirement of additional testing apart from integration and unit testing. This additional testing entails data and model testing on top of integration and unit testing.

The fourth difference is in the deployment process. Apart from fast deployment, the goal is to be able to re-run the deployment based on the signals that appear during production. 

Another difference is the monitoring for additional metrics. The traditional metrics used to determine health, traffic, and memory in DevOps are also used in MLOps. However, in the case of MLOps, additional metrics are also required such as prediction quality and model performance, etc.

Do You Need MLOps for Your Startup or Enterprise Business?

It depends on the business and resources. If the data does not change frequently and only needs an update once or twice a year, manual processes might suffice for such businesses. For other businesses where the data is updated a few times a month such as insurance risk and disaster risk-related business use-cases, a certain degree of machine learning automation will be beneficial. 

There is also the case of businesses where data is changing very frequently in which case a full MLOps should be the only way forward. For example, in the case of spam detection, the latest data in spam is required and models need to be re-trained in a full feedback loop pipeline. These factors, among many others, determine your need for MLOps.

Should You Automate the Entire MLOps Pipeline or Parts of It?

MLOps is all about monitoring and automation of the pipeline and should be carried out as required. The most frequent part of the business should be automated instead of automating the entire pipeline. Businesses that deploy once a year would not benefit from automating the entire process; however, if your deployment is fairly regular, then MLOps should be practiced.

ML Pipeline Vs CI/CD Pipeline: what are the differences and similarities?

Continuous Integration  (CI) and Continuous Deployment (CD) pipelines are relatively the same as MLOps pipelines with a few additional components. CI/CD pipelines are no longer merely testing and validating code and components but also testing and validating data schemas and models as well. It is no longer a single software package or a service but an entire system that should automatically deploy another service. Consequently, it becomes a machine learning training pipeline that, upon prompt, deploys another service, unlike traditional continuous deployment which is well-defined and linear. The continuous training component is a new feature that the MLOps pipeline exhibits. Another distinguishing factor is the continuous model monitoring in production.

What should we understand from Model or Concept Drift?

The complete performance assessment of a machine learning model in online systems is a challenge and therefore requires an offline model to train the data on a specific use-case and generate a prediction model. However, the offline model performance deviates from the online model performance implying that the model has “gone still” and needs to be re-trained on fresh data. This phenomenon is referred to as the concept drift and requires continuous training in ML systems.

How to Monitor the MLOps Deployment?

In MLOps deployment, apart from the aforementioned traditional metrics, health, memory, time variation, and latency (i.e. the time it takes the model to make a prediction) are also monitored. The throughput metric in MLOps deployment is another important metric as it deliberates on how many examples the model can predict in one instance of prediction. Yet another important metric is data schema skew because data changes need to be monitored with a great deal of attention in ML.

As a Startup Founder, Can I Hire DevOps Engineers to Work as MLOps Engineers?

It is believed in the technology community that either a  machine learning engineer or DevOps engineer should work as an MLOps engineer. Therefore, it could be that the DevOps engineer learns the machine learning systems to become an MLOps engineer and vice versa. However, without a thorough understanding of machine learning that a data scientist would have, solely a DevOps engineer might miss out on important factors, and similarly, without software engineering knowledge, the data scientist might miss out on important deployment details. So, this is turning into a newly rising field called machine learning software engineer with skills to develop basic machine learning models as well as testing and monitoring the model in deployment.

How To Determine if an ML Model Is Ready to Be Released?

As mentioned above, an ML model is trained and deployed for an offline performance evaluation based on a given data set. If the model performed well offline, it would be deployed in production. It is a very basic version of the deployment process. However, a more robust approach is to perform AB testing on the ML model that has been deployed in production, determine its performance, and update the model during production. The model is then constantly evaluated to determine its performance and readiness as opposed to previous practice.

How Do Tech Giants Use MLOps?

For instance, Google has an open-source policy and uses most of the internal libraries or frameworks. There are different levels of software that one can use based on the required sophistication. Another big part of Google culture is Auto ML which performs the activities typical to a machine learning team. The data is injected in Auto ML which automatically trains data and offers a shortcut and accelerates typical machine learning projects and their tasks.

TensorFlow data is also used by Google to monitor production data quality and to compute the basic statistics. Interactive charts allow for visualized inspection and comparison to monitor any concept drifts. On top is a Cloud Composer which essentially orchestrates all the aforementioned functions starting from data exploration and collection to model serving and monitoring among others. In a basic setup, Cloud Composer acts as the orchestrator. For more advanced processes, KubeFlow is available which is a Kubernetes-native machine learning toolkit. It can build and deploy portable and scalable end-to-end machine learning workflows based on containers. Another advanced tool is TFX, recently open-sourced by Google, which is a configuration that provides components to define, launch and monitor TensorFlow-based models and performs model training, prediction, and the serving.

Step by Step Guide for Companies to Adopt MLOps

Many Cloud Service Providers (CSPs) offer abstraction for MLOps. As mentioned earlier, Google Cloud Services offer Google ML Kit. Similarly Azure offers Azure ML service, and Amazon offers Sage Maker. These abstractions allow data scientists to focus on business logic rather than production-related problems.

To Watch the Complete Episode, Please Click on This YouTube Link:

https://www.youtube.com/watch?v=d19JfKF5Y38&t=274s

Introduction

While cloud services have made application development more scalable and flexible, startup founders have been reluctant to adopt cloud-native environments due to inherent security risks. 

This blog aims to answer the questions that startup founders have about best practices in cloud-native application(s) security, how these practices are implemented and overseen in the cloud, and what benefits they can provide for a startup. You can sleep soundly knowing that all aspects of your application development security are covered.

What Is the Difference Between Traditional Application Security and Cloud-Native Security?

In traditional applications before the cloud, development and security requirements were fulfilled by teams of developers. Infrastructure management was managed by a different team but end-to-end securing of the network, compute, and storage was ensured by developers. This required more manpower, processes, and policies such as setting up hardware firewalls, DDOS mitigations on-prem, etc. 

With the advent of the cloud and its security, the only important thing is the context of operation for your application. Surely, the cloud has a lot to offer in terms of flexibility along with a range of services and infrastructure. However, these features also enlarge the attack surface area in order of magnitude as compared to traditional application security. In light of this, understanding the context in which your application is operating becomes crucial because now you need to model your application while understanding its input/output patterns, interaction pieces, etc. Your applications may have inputs from event streams, cloud storage changes, and databases, potentially introducing additional attack vectors. Hence, you must secure the platform and the infrastructure that enables your application. Only this kind of thorough and continuous hardening would implement application security in a cloud-native environment.

Does This Imply That the Advent of the Cloud Has Made Security More Difficult Rather Than Easy?

It hasn’t made security difficult per se; however, the advent of the cloud has instituted the concept of Shared Responsibility.

As mentioned earlier, traditional application development, infrastructure, and security were managed by on-prem IT teams but with the cloud, some of those responsibilities have been outsourced to the Cloud Service Providers (CSPs) who are responsible to secure their infrastructure and software layers that they operate on, formally known as Inherited Security Controls, that covers the security from CSPs’ side. 

For the startups using these services, the responsibility to secure their domain lies with themselves. For example, configuration management, identity and access management policy, securing databases, data encryption in-transit and in-rest, etc. 

CSPs provide you with tools to secure your domain. The ultimate responsibility lies with you in the Shared Responsibility Model to administer the right configurations and apply the right guard rails at each layer. 

Cloud has not made security difficult. Instead, it has divided the responsibility between the service providers and startups to ensure robust and secure applications. 

How Would You Describe a Secure Software Development Lifecycle?

To secure any software development lifecycle (SDLC), having third-party APIs or open-source libraries is no longer enough. Instead, security has to be catered to at every stage of the software development lifecycle. It is ideal to have a secure automated testing pipeline built into all stages of SDLC of your application.

For cloud-native security, however, you need to protect all of the assets that you have deployed on the cloud. This requires, first of all, understanding the context and substrate your application operates in e.g. containers or VMS. Ensuring security at Operating Systems (OS) and application levels requires administering good principles and policies. 

Firstly, Static Application Security Testing (SAST) which is built into the CI/CD pipeline allows developers to statically analyze vulnerabilities in their source code and the third-party libraries. 

Secondly, Dynamic Application Security Testing (DAST) also known as penetration testing is carried out once the application has been deployed. It allows the developers to scan the ports and IPs of your space, anything that is open and vulnerable. Once identified, you can take measures to mitigate it. 

Thirdly, Interactive Application Static Testing IAST resembles DAST in operating dynamically. However, the goal of IAST is to find out runtime vulnerabilities in a runtime scanning infrastructure that continuously scans and monitors your environment. 

Lastly and more recently, Runtime Application Self-Protection (RASP) where your applications are built in a way (or are supported by services) that essentially detect these issues that happen at runtime and then do self-mitigating actions, generate real-time alerts if something suspicious has happened. For example, say there’s a DDoS going on, how to detect and automatically remediate it. This is what falls under RASP. 

All of these measures together contribute to creating a secure development lifecycle for an application.

What Are the Best Practices for Secure Application Development and Deployment?

For ironclad secure application development, Identity and Access Management (IAM), encryption, threat monitoring response, data privacy and compliance, and automated security testing are major boxes that need to be checked irrespective of what cloud you are going to deploy online. 

For Identity and Access Management, the principle of Zero Trust is followed which means you should not trust anyone. You have to model your application with knowledge of who is going to access it, and what is the level of privileges they have. CSPs give you tools to implement these identities and access management.

The importance of data encryption is undisputed. There are different types of data encryptions namely, in-transit, in-rest, and in-use. In-transit, in-rest and in-use encryptions help protect your data during communication, when data is stored/is at rest, and when data is operated on, respectively.

For threat monitoring and incident response, you need to have an infrastructure in place that continuously scans and monitors for threats. Once a threat is detected, you must have a strategy in place to respond to that threat. If these strategies are an afterthought for a startup, they can expect a huge blast radius when they are hit with a threat. So, detection and response to security-related incidents are critical. 

Data privacy and compliance deal with different localities, regulations, and laws that apply to these local sites. The data privacy and compliance mechanisms in your application have to identify and address these differences depending on your location and the location of your CSPs.  

The necessity of Automated Security Testing cannot be stressed enough and it implies that ideally at each stage of SDLC, you have all of the controls i.e. SAST, DAST, and RASP have been built in the CI/CD pipeline. 

Lastly, securing your APIs and secure modeling of the communications patterns that your application exhibits is really important as well. 

Is Monolithic Architecture More Secure Than Microservice Architecture?

Monolithic architecture is a single-tier design with security parameters built around that tier. This might make securing that one tier relatively easy, but monoliths have their disadvantages. For example, you cannot innovate fast enough, or add new features efficiently. Whereas, microservices architecture can decouple the responsibility of each service and is iteratively able to improve and consequently upgrade your product fast. Modern architecture requires you to have that very fast prototyping and development and no matter how secure monoliths may be, microservices are the future. 

Does all the data need to be encrypted?

Ideally, yes! But, of course, there are costs associated with it.

When you encrypt data at rest, it requires additional resources in terms of time and money for ensuring the cryptographic validity of your data. The overhead extends to performance as well because encryption requires decryption. So, your encryption scope is defined by the context your application is operating in. You have to strike a balance between performance and data security. 

How Is DevOps Different From DevSecOps?

DevSecOps is just a subset of DevOps. The concepts differ in the notion that DevOps is purely just CI/CD and IaC allows you to rapidly develop applications and deploy them whereas DevSecOps deals with automated security testing that is built into the pipelines resulting in development and security operations. On one hand, DevOps facilitates the configuration of your IaC and on the other, DevSecOps facilitates the additional guardrails built for security-specific functions. That is where the industry normally decouples the concepts of DevOps and DevSecOps in the application’s continuous integration and deployment. 

A right DevOps pipeline must have security built-in, it can’t be an afterthought. DevSecOps directly targets the security-specific things in a DevOps pipeline and a DevOps pipeline is essentially just the application’s continuous integration and deployment.

Does Cloud-Native Security Mitigate Insider Threats?

Insider threats are a vast and interesting domain as there are models that can be used to predict them but mostly they are very hard to find. Insider threat means that you either have some unauthorized user who has their privilege elevated or there is some malicious insider who essentially has access to data and has somehow exported it out.

Cloud-native security provides you with the tools to mitigate insider threats. For example, you can set up the right access identity and access management and follow the principle of least privilege where the people who are supposed to have access to that data are the ones who do, and then you can also establish parameters around services. You have, at your disposal, tools to implement these controls but insider threats will still happen and you need to have a robust incident management detection and response strategy to deal with that. 

For payment industry solutions, there is a specific standard called Payment Card Industry Data Security Standard (PCI DSS). It has checklists of principles that are implemented for any application that provides a payment solution. For example, it includes having active firewalls, the right IAM, the encryption in place, and protecting the cardholder. Any application that is being built with a payment feature has to abide by these standards. What is important to note here is that with the increasing amount of CSPs in the market, all of them ensure that their services are PCI DSS compliant. Still, as an application developer or a startup founder, you have to make sure that you have the right vetting in place to guarantee standard compliance. 

Is Cloud Security Offered by Cloud Service Providers (CSPs) Enough for Your Application?

CSPs will ensure their layers of software, hardware, and infrastructure are secure and any breach in those layers will be their responsibility. Keeping Shared Responsibility Model in mind, the second layer is the domain of the startup using the services. Therefore, securing that layer is the responsibility of the startup itself. For example, robust IAM strategy, patch management, and configuration management need to be secured by your company to deter exploits. 

How Do I Secure Hybrid and Multi-Cloud Environments?

Hybrid environments are a challenge because there are different tools for on-prem and cloud to implement security. Ideally, there would be a single dashboard to store, operate, and monitor such environments. CSPs have been increasingly providing features for a seamless experience across different environments. For example, Google has Anthos where you can run the Kubernetes engine on-prem as well as on the cloud at the same time and have a single pane of management across them. Similarly, AWS has AWS Outposts which allows you to use AWS stack and services on-prem as well. The important thing to note here is knowing the context; what are the communication patterns that exist on-prem, what are the communication patterns that exist on the cloud, how do you secure them, can you have one solution that spans across both or will you have to have different solutions that logically create a single security interface but are decoupled in physical implementation? 

How Is Application Security Guaranteed in a Multi-Geo Environment?

In the shared responsibility model, the third layer is customer-specific and depends on the operation site of your application. For example, the Global Data Protection Regulation (GDPR) standards and protocols need to be followed for customer data stored in Europe. According to their data regulations or their environmental regulations, this data cannot move out of said location. So if you are developing an application that spans different geographies, the tools to implement security guardrails are available but a robust strategy to develop an end-to-end secure application that operates in different jurisdictions and has different governing laws has to be formulated by the startups.  

What Should Be the Course of Action in Case the Breach Has Happened?

There are a few standardized steps, for example, you have to capture events in your system and analyze them. You need to know not only the source IP of the malicious request but also the destination IP, the protocol, and the fields inside these requests. There are different ways to address a breach. Some startups build in-house solutions that define the rules that are applicable for such events. Analyzing and mapping these events to normal traffic generates alerts. The incident response team has automated pipelines in place to identify and launch actions that address said alerts. Generally, continuous threat monitoring and incident response are one of the very critical functions of having security in this day and age. 

In Proactive Security Strategies, Does Artificial Intelligence (AI) Play a Role?

Proactive security detection means automated security testing throughout your SDLC. Tools are available that allow you to continuously scan your Static Source Code to find out vulnerabilities and then automatically suggest solutions to mitigate those vulnerabilities. Similarly, Dynamic Code Analysis and Runtime Code Analysis allow you to map traffic patterns so that you can identify any anomaly. AI models help you predict these patterns and identify anomalies. This, in itself, is a vast domain and AI plays a role in contextualizing the full security problem.

To Watch the Complete Episode, Please Click on This YouTube Link:

https://www.youtube.com/watch?v=hPT1Zj3tNV4&t

Enhanced user experience is one of the top goals for every SaaS provider. How do you ensure that your customers are experiencing the value you intended your applications to deliver? 

“Customer experience is the next competitive battleground.”

– Jerry Gregoire (the founder and Chairman of Redbird Flight Simulations, Inc.)

Having real-time, non-real-time and partially real-time metrics at your fingertips may help you to stay ahead of expected and unexpected challenges regarding your service performance, business, and operations.  

Why do startups need to monitor their well-tested applications?

Earlier, the software was delivered (or “shipped”) to the customers through a CD and that was the extent of it. The job of the software service provider was completed. However, since the advent of the SaaS and the cloud services, the software developers have visual access to more information about the delivered application, for instance, how customers are using it, which feature is popular with the user, which feature is unpopular among the users, and much more.

This allows the SaaS providers to leave the hardware mindset behind, they no longer have to ‘build it to never break’. Instead, the application monitoring allows SaaS providers to make customer satisfaction an iterative process for a product. Why?  Because application monitoring allows you to not only track the performance of your product in the customer environment but also the oversight of operations and some valuable business insights that you can leverage to enhance your revenues. 

What are the different categories of metrics used to monitor such applications?

Broadly, the application monitoring metrics are divided into three categories.

The first and most important one is operational metrics. Generally, it covers the health of your services, all the related underlying microservices, and the interaction of the service in its environment. Operational metrics are usually real-time metrics that notify SaaS providers as soon as an application or any feature in the application ceases to function.  

The second category of metrics is business metrics which determine if your service is offering the value that you designed it to offer. Business metrics can be real-time metrics but they usually are not. They are measured over weeks and months to identify a trend. 

The third type of metric is performance metrics. An application may be running and providing the value it was meant to, but it may be taking a long time to load, or TCP handshake is taking longer than expected, such measurements are defined by the performance metrics of the application. Performance metrics are semi-real-time metrics. 

What are some good examples of operational metrics?

The basic metrics that SaaS providers can use for monitoring applications like operationality, CPU memory, or I/O usage are offered by the cloud for free. These are also metrics that don’t require any instrumentation in your code.

There are instances where metrics are measured through the use of instrumentation installed in your application. If the code crashes, these instrumentations will generate an alert for the undesired event across multiple microservices. 

Metrics that are threshold-based and can generate a notification and/or when the threshold is breached are also used in operations depending on your services. 

Operational metrics are layered in that order that you start small and add sophisticated metrics over that foundation. 

What are some top-tier performance metrics for application monitoring?

Page load time is a good example of how you measure the performance of your service. Similarly the time it takes the customer to submit a form hosted on your website is another. The time it takes for my ride-hailing app to enter an address and then the time difference to when the cab driver receives the notification are all performance measurements that vary depending on your service. 

For more sophisticated monitoring, metrics such as upon notification of an error, observing performance at a more granular level like TCP handshake time and SSL time, etc. are incorporated and you gradually build on top of simpler metrics. 

Does microservices architecture pose more complexity compared to a monolithic architecture?

Microservices may be harder to build but are easier to monitor because the metric that is experiencing error will have the issue limited to that microservice alone. 

Monolith applications are easier to build but are harder to monitor, simply because if something is rendered defective, debugging and addressing the problem can be very time-consuming and will affect every aspect of your business. 

What are your recommendations regarding an alert system?

The Cloudwatch monitoring and observability solution for application and infrastructure by Amazon is a very powerful system. There are multiple alerting tools like email notifications, SNS, and SQL notifications that are built into it. It also offers multiple ticketing as well as communication system integrations, on-call tools, PagerDuty, and much more. 

Other than that, Google Cloud Monitoring,  DataDog, and New Relic are a few SaaS providers of Monitoring Solutions. 

How do monitoring solutions help operations teams with correlation, triage, and root cause diagnosis?

As a SaaS service provider, you ought to know about a problem in your service or application before or when the customer complains. One application is supported by multiple microservices and identification of the performance issue and knowing where to find that issue before you can resolve it, is only possible through metrics for monitoring applications. 

Once you have identified the problem and its source of a performance bottleneck, you can start fixing that problem to bring the application back to its original functional state and that’s how triage, correlation, and root cause diagnosis help you with identification, mediating the problem, and keeping your service functional.  

Are there monitoring solutions where you are proactively looking and might foresee an event?

As a SaaS-based startup, your software stack is missing a key monitoring aspect if your system was unable to alert you hours ahead of your customer. Your customer informing you about an error should be the last resort. Usually, this last resort is seen in instances where SaaS providers missed or misjudged some metric and deemed it irrelevant once it broke in the customer environment and you weren’t notified. SaaS providers need to carefully choose their monitoring deck to avoid this oversight and monitoring needs to be done proactively rather than reactively. 

How much should the cost be for any application monitoring solution?

Cloudwatch, on-call rotation systems, and other metrics are not super expensive but can get expensive as your application scales and requires more and more components.  The human resources may be expensive, however, the actual dollar cost of setting up monitoring systems is generally pretty low. 

How do we monitor API endpoints?

For APIs, I always suggest canaries, simply because it is the easiest, fastest, and painless way to monitor your API endpoints. Every public-facing API should have multiple canaries that are continually testing it. There are numerous tools available, in the market, by cloud providers and third-party providers for testing API endpoints, in terms of security testing, function, and performance testing. 

Cloudwatch uses the concept of synthetics that allows you to set up a synthetic stack in another region from where you can continuously test your public API endpoints. 

How does a startup founder go about setting up a monitoring system and which tools are the best ones for it?

As a startup founder, it is easy to fall for the idea that more data means more results. I would caution against it and instead suggest starting small. For instance, start with CPU memory, focus on mechanisms and processes, and make a trial of when things might break. Install solutions for those points. Start with a few metrics preferably that can fill in one single screen and then focus on your mechanisms, your backend system, your paging, and the culture of operational excellence rather than the metrics themselves. Once you have achieved that successfully without oversights, evolve from there. 

What is the difference between agent-based and agentless monitoring?

In agentless monitoring, an agent is not required to be installed in the system for you to be able to monitor it. Systems that already exist emit those metrics automatically. Agent-based monitoring requires an agent to be installed when you integrate with an SDK and that agent collects all the desired metrics and emits them to your application backend. I recommend agentless monitoring but it can only take you so far. The agent-based approach provides much deeper insights. Take application performance monitoring as an example. Here, agent-based monitoring can allow you to passively monitor dom objects and measure the page load time via a short script that allows you to observe which part of the TCP handshake is taking longer, and so forth. 

This depends on how deep you want to observe the operations or performance and also on the feasibility of inserting an agent-based solution. 

What good vs. bad monitoring practices do you see in the market and what does the future look like for Monitoring Solutions?

For now, there are two strategies in the market: the new way of shipping software with monitoring and observability services vs. the old way of shipping systems that never break. For me, technology is easy, the people are the hardest part of the puzzle. But I have noticed that startups are stuck in legacy practices of not having visibility to their customer environment and their service performance. 

And as far as the future is concerned, the practice of monitoring your application is the future, where you can measure performance metrics, supervise the operational metrics of your application and observe business metrics for what features are providing value and which are not. All these metrics provide you with significant data to make agile decisions for your startup. All successful companies today, as well as in the future, will have these metrics and will be able to look each other in the eye and say we made a mistake and we’re going to change it because the data tells us otherwise. I think such courageous teams are winning today and they will continue to win in the future as well.

Infrastructure as Code (IaC) is a dynamic and automated foundation for DevOps. But, as a startup founder, how much do you really know about IaC? 

We invited Abdullah Shah, Senior Software Engineer working with the Cloud Security Engineering Team at Salesforce, to enlighten us about IaC. Here is how it went down:

Why is there a need for IaC? What trouble will companies face if they don’t embrace IaC?

Without IaC, the IT infrastructure is configured, operated, and maintained manually. Historically, applications were developed using legacy IT infrastructure i.e. servers, storage, and compute. All these functions were provided using bare metal hardware in an on-prem environment. The configuration, operation, and maintenance were manually performed which meant high rates of human errors, delays, and demand for shedload members on the IT team. In addition, these manual processes exhibited lack of standardization of procedures and ineffective documentation, if at all. Collectively, this resulted not only in Environmental Drift but overall an inconsistent environment for application development and created even more challenges during scale-ups. 

Explain to us the concept of IaC, and why should the companies work with it? 

The irregularities we discussed in the absence of IaC, necessitate a more sophisticated solution for infrastructure to develop products. Infrastructure as Code (IaC) is that very revolution. 

IaC, in contrast to the legacy manual model, is a more descriptive model which allows the configuration, provision, and management of infrastructure using machine-readable files.

With automated Infrastructure, configuration, operations, and maintenance functions are now performed using scripts. As a result, IaC gives you performance consistency, monitoring accountability, operations agility, scale-up flexibility,  audit transparency, software security, and a much faster development cycle. Companies that have embraced IaC benefit from reduced operations costs and rapid time to market new products, services, and features. All in all, higher revenue and more satisfied customers. 

As a Start Up founder, what steps do I need to take to embrace IaC?

I would advise promptly embracing IaC or at least outlining the roadmap to immediately focus on embracing it. 

The first step is to evaluate the infrastructure requirements for the products and/or services you offer. Secondly, with multiple options available, make a categorical decision on your tech stack according to your product. 

Startups usually want to build and push products to the market rather quickly since they believe it’s merely prototype(s). However, I would encourage you to create a solid foundation with the right IaC principles and to implement IaC from the ground up. With a strong footing, scaling beyond the first 2- 3 servers will become streamlined and efficient with IaC. 

Is IaC required to have DevOps in a Start Up? How are they related? 

DevOps and IaC go hand in hand. One would not exist without the other. Although all companies apply the DevOps principles to varying degrees, the most popular is the Shift Left approach which is synonymous with the service ownership model. The idea is that the developers in IT teams are not working in silos but collaboratively, to create a holistic view of the entire application development lifecycle. In this spirit, developers are not only responsible for application development but also for creating the right infrastructure for the operations and deployment of the application as well. This means that the responsibility of coding has been fanned out among the IT team members. Testing and monitoring roles have been shifted left to the developers and all of this is enabled by IaC. 

Do I need to test and monitor IaC?

There is no substitute for testing and monitoring IaC; it practices more stringent test requirements. Infrastructure can be automated flawlessly with the use of correct IaC templates and avoiding misconfigurations. An array of testing tools are available for you to choose but the fundamental notion of testing being critical, for IaC, cannot be overstated.

With QA functions disappearing in pure DevOps culture, who would carry out these stringent testing and monitoring?

In pure DevOps, CI/CD has automated IT testing and operations which in turn accelerates the process of application deployment exponentially. This results in continuous updates to your application and infrastructure. Now, if you don’t have automated testing, you are in a heap of trouble. At this rate of deployment, humans can not keep up and companies must implement automated testing strategies throughout the application development supply chain. 

How would you address the IaC related fear of automation, in industry?

The idea is simple. If you anticipate failure, you can prepare for it and then mitigate it. The preparation you need is to implement a robust testing strategy and it is equally important to have a control feedback loop to continuously audit the strategy and improve. The ultimate goal is the provision of the right environment that doesn’t miss any build failures. Failures, caught and addressed in the next loop, will allow streamlined product deployment moving forward. 

Explain the difference between declarative and imperative approaches for us.

The difference is that Declarative is a functional approach and Imperative is a procedural one. Declarative IaC approach answers the question ‘What is the preferred state’ and Imperative IaC approach answers the question ‘How is the preferred state achieved’’. Since it is critical to maintain your infrastructure in the desired state, it is recommended to use a declarative IaC approach. Imperative IaC approach relies on control loops and conditional statements which can become a complex endeavor. 

Some supporting tools are available that create an imperative layer over the declarative model to provide an abstraction. Pulumi, for instance, is one such tool that is self declarative and can provide an imperative layer. Amazon CDK and Terraform are other examples of such tools that provide the best of both approaches. 

Which of these approaches in your opinion can help the companies with their tech debt?

Traditional practices slow down the application development cycle and can lead to technical debts. In unpredictable cases e.g. immediate customer requirement, badly written code, or new feature request, the right automated testing strategies are your only way out of incurring technical debt. That is exactly what IaC promises. It creates those guard rails around your processes that reduce technical debt. 

Mutable and Immutable infrastructures, which one would you recommend? 

During application development, changes are inevitable. Your infrastructure will need to be scaled up or down, updated, and/or patches will have to be applied. Post-deployment, if the infrastructure can be changed, it is mutable infrastructure and vice versa. In case of immutable infrastructure, the changes are rolled out on a replica new machine before the old immutable infrastructure is taken down. There are multiple ways to go about it. However, when in doubt, go with mutable infrastructure. 

In terms of tech stack, what are the best tools to implement IaC?

There is a spectrum of choices available, as discussed earlier. Amazon CDK, CDKFT, CloudFormation, Terraform, and Pulumi are all tools used to implement IaC. I suggest democratizing the decision among developers, SREs, and stakeholders since tech stack is not only meant for IaC but the entire application development pipeline is orchestrated. For developers, you have version control using Git; for operations CI/CD, AWS pipeline, GitOps; for code build service, there are custom Jenkins tools. Argo CD is a popular tool that operates specifically with Kubernetes-based deployments while Spinnaker allows you to deploy custom IaC technologies. However, the decision depends on your use cases and what is required to implement them.

My recommendation for IaC, is Terraform, as they offer one of the best tools, have an early mover advantage, a vibrant community, simplified syntax, and well-written documentation.

Does IaC help with the provisioning? 

For Sure. IaC, much like software development, is version controlled in a centralized repository. Any update or change in features will be validated, tested, built, and pushed to required registries automatically. These automatic processes present a holistic development pipeline. All these tools come together to facilitate everything from integration to deployment and provisioning is definitely one aspect of the whole picture as well. 

About IaC On Cloud Vs IaC On-Prem; your thoughts?

On-Prem infrastructures are heterogeneous when it comes to provisioning identity services, secret services, and/or network security. Limited automations can be performed on On-Prem environments. Most services have to be custom-built and are not scalable. On-Cloud, on the other hand, is standardized in service provision. Infrastructural resources become more flexible to scale up or down as required. Documentation best practices, abstractions at hand, repeatable and predictable environment are some of the factors that put IaC On-Cloud in a league of its own and hence offer more value. 

What IaC profiles should I look for, as a StartUp founder? 

Broadly speaking, for pure DevOps, Developers and Site Reliability Engineers (SRE) with the mindset of service ownership model are increasingly in-demand skills in Silicon Valley and across the world as well.

Tell us about the best practices for IaC in the industry?

To build an application infrastructure specifically in the agile world of IaC, the best practices require a set of foundational services at the periphery as well. For your company, best practices mean the acquisition of full package of foundational services i.e. Compute, storage, database, networking, security, all of it. 

What does the future hold for IaC?

IaC has already saved a lot of money for different companies. It has sped up the process of software development from integration to deployment and ultimately delivers value to the customers. IaC helps create and serve those customers in a very fast, agile, and repeatable way. 

In terms of the future, we already know manual infrastructure provision can not scale or perform at the same pace as IaC. Hence, lost customers and technical debt are natural outcomes. To enable those fast, repeatable deployments, and product iterations, IaC is the inevitable future and I very strongly believe that companies are and will implement full robust, orchestrated infrastructure, and pipelines at the heart of their businesses. 

To Watch the Complete Episode, Please Click on This YouTube Link:

https://www.youtube.com/watch?v=d19JfKF5Y38&t=274s