Category: AzureDevOps

Azure Architect Expert Study Notes

The following is a quick and dirty list I made for the Architect exams so that I could read them quickly before the exam itself. This is mostly for the AZ-302 but good to know regardless of what exam your doing.

  • Blob Storage is NOT for storing Virtual machine vhd files, blob storage is for block blobs and append blobs and not page blobs)
  • You can use Traffic Manager to sit above 2 virtual machines and register endpoints, if one of the region goes down the other stays up.

The following traffic routing methods are available in Traffic Manager:

  • Priority: Select Priority when you want to use a primary service endpoint for all traffic, and provide backups in case the primary or the backup endpoints are unavailable.
  • Weighted: Select Weighted when you want to distribute traffic across a set of endpoints, either evenly or according to weights, which you define.
  • Performance: Select Performance when you have endpoints in different geographic locations and you want end users to use the “closest” endpoint in terms of the lowest network latency.
  • Geographic: Select Geographic so that users are directed to specific endpoints (Azure, External, or Nested) based on which geographic location their DNS query originates from. This empowers Traffic Manager customers to enable scenarios where knowing a user’s geographic region and routing them based on that is important. Examples include complying with data sovereignty mandates, localization of content & user experience and measuring traffic from different regions.
  • Multivalue: Select MultiValue for Traffic Manager profiles that can only have IPv4/IPv6 addresses as endpoints. When a query is received for this profile, all healthy endpoints are returned.
  • Subnet: Select Subnet traffic-routing method to map sets of end-user IP address ranges to a specific endpoint within a Traffic Manager profile. When a request is received, the endpoint returned will be the one mapped for that request’s source IP address.

App Service plan pricing Tiers

There are a few categories of pricing tiers:

  • Shared compute: Free and Shared, the two base tiers, runs an app on the same Azure VM as other App Service apps, including apps of other customers. These tiers allocate CPU quotas to each app that runs on the shared resources, and the resources cannot scale out.
  • Dedicated compute: The Basic, Standard, Premium, and PremiumV2 tiers run apps on dedicated Azure VMs. Only apps in the same App Service plan share the same compute resources. The higher the tier, the more VM instances are available to you for scale-out.
  • Isolated: This tier runs dedicated Azure VMs on dedicated Azure Virtual Networks, which provides network isolation on top of compute isolation to your apps. It provides the maximum scale-out capabilities.
  • Consumption: This tier is only available to function apps. It scales the functions dynamically depending on workload. For more information, see Azure Functions hosting plans comparison

Logic Apps

TO enable high throughput on a Logic App you can go to workflow settings and then choose High Throughput and click ON, this allows up to 300,000 executions every 5 minutes.

App Service Plans

The basic App Service Plan doesn’t support auto-scaling


Create a Linux virtual machine with Accelerated Networking

To create a Windows VM with Accelerated Networking, see Create a Windows VM with Accelerated Networking. Accelerated networking enables single root I/O virtualization (SR-IOV) to a VM, greatly improving its networking performance. This high-performance path bypasses the host from the datapath, reducing latency, jitter, and CPU utilization, for use with the most demanding network workloads on supported VM types. The following picture shows communication between two VMs with and without accelerated networking


Azure Migrate

Migrate databases to Azure with familiar tools

Azure Database Migration Service integrates some of the functionality of our existing tools and services. It provides customers with a comprehensive, highly available solution. The service uses the Data Migration Assistant to generate assessment reports that provide recommendations to guide you through the changes required prior to performing a migration. It’s up to you to perform any remediation required. When you’re ready to begin the migration process, Azure Database Migration Service performs all of the required steps. You can fire and forget your migration projects with peace of mind, knowing that the process takes advantage of best practices as determined by Microsoft.

Note: Using Azure Database Migration Service to perform an online migration requires creating an instance based on the Premium pricing tier.


Types of storage accounts

Azure Storage offers several types of storage accounts. Each type supports different features and has its own pricing model. Consider these differences before you create a storage account to determine the type of account that is best for your applications. The types of storage accounts are:

  • General-purpose v2 accounts: Basic storage account type for blobs, files, queues, and tables. Recommended for most scenarios using Azure Storage.
  • General-purpose v1 accounts: Legacy account type for blobs, files, queues, and tables. Use general-purpose v2 accounts instead when possible.
  • Block blob storage accounts: Blob-only storage accounts with premium performance characteristics. Recommended for scenarios with high transactions rates, using smaller objects, or requiring consistently low storage latency.
  • FileStorage (preview) storage accounts: Files-only storage accounts with premium performance characteristics. Recommended for enterprise or high performance scale applications.
  • Blob storage accounts: Blob-only storage accounts. Use general-purpose v2 accounts instead when possible.

Azure Functions ARR affinity

If you create azure functions as part of the Basic app service plan, you can enable ARR Affinity which basically allows support for sticky sessions.

Azure App Service Access Restrictions

Access Restrictions enable you to define a priority ordered allow/deny list that controls network access to your app. The list can include IP addresses or Azure Virtual Network subnets. When there are one or more entries, there is then an implicit “deny all” that exists at the end of the list.

Auto Swap Staging Slots (Auto Swap isn’t supported in web apps on Linux.)

VNet Peering – connecting VM’s within the same Azure Region

Global VNet Peering – connecting VM’s across Azure Regions

Choose between Azure messaging services – Event Grid, Event Hubs, and Service Bus

Comparison of services

Service Purpose Type When to use
Event Grid Reactive programming Event distribution (discrete) React to status changes
Event Hubs Big data pipeline Event streaming (series) Telemetry and distributed data streaming
Service Bus High-value enterprise messaging Message Order processing and financial transactions

Event Grid

Event Grid is an eventing backplane that enables event-driven, reactive programming. It uses a publish-subscribe model. Publishers emit events, but have no expectation about which events are handled. Subscribers decide which events they want to handle.

Event Grid is deeply integrated with Azure services and can be integrated with third-party services. It simplifies event consumption and lowers costs by eliminating the need for constant polling. Event Grid efficiently and reliably routes events from Azure and non-Azure resources. It distributes the events to registered subscriber endpoints. The event message has the information you need to react to changes in services and applications. Event Grid isn’t a data pipeline, and doesn’t deliver the actual object that was updated.

Event Grid supports dead-lettering for events that aren’t delivered to an endpoint.

It has the following characteristics:

  • dynamically scalable
  • low cost
  • serverless
  • at least once delivery

Event Hubs

Azure Event Hubs is a big data pipeline. It facilitates the capture, retention, and replay of telemetry and event stream data. The data can come from many concurrent sources. Event Hubs allows telemetry and event data to be made available to a variety of stream-processing infrastructures and analytics services. It is available either as data streams or bundled event batches. This service provides a single solution that enables rapid data retrieval for real-time processing as well as repeated replay of stored raw data. It can capture the streaming data into a file for processing and analysis.

It has the following characteristics:

  • low latency
  • capable of receiving and processing millions of events per second
  • at least once delivery

Service Bus

Service Bus is intended for traditional enterprise applications. These enterprise applications require transactions, ordering, duplicate detection, and instantaneous consistency. Service Bus enables cloud-native applications to provide reliable state transition management for business processes. When handling high-value messages that cannot be lost or duplicated, use Azure Service Bus. Service Bus also facilitates highly secure communication across hybrid cloud solutions and can connect existing on-premises systems to cloud solutions.

Service Bus is a brokered messaging system. It stores messages in a “broker” (for example, a queue) until the consuming party is ready to receive the messages.

It has the following characteristics:

  • reliable asynchronous message delivery (enterprise messaging as a service) that requires polling
  • advanced messaging features like FIFO, batching/sessions, transactions, dead-lettering, temporal control, routing and filtering, and duplicate detection
  • at least once delivery
  • optional in-order delivery

Notification Hubs

Has an SLA of 99.99% on the Basic and Standard tiers

RPO – Recovery Point Objective – The amount of data loss if a recovery needs to be done

RTO – Recovery Time Objective – The amount of time it takes to complete a recovery or restore

Azure Backup

Recover Points

  • Application Consistent – Here the backup takes into consideration any pending i/o operations and memory content operations. This allows the application to start in a consistent state after recovery.
  • File System Consistent – This provides a consistent backup of disk files. Here the application needs to maintain its own mechanism to manage its consistency.
  • Crash Consistent – Happens when the VM Shuts down at the time of the backup. Data exists on the disk at the time of the backup, but not guarantee on the disk consistency.

Azure Backup is good for retention periods of days, weeks, months and eve years.

Virtual Machines SLA’s

One VM = 99.9% availability

Two or more VM’s in an Availability Zone = 99.99% availability

Two or more VM’s in an Availability Set = 99.95% availability

Availability Zones

Within 1 Region you may have 2 availability zones

So this can mean 2 Availability Zones each having 2 data centres.

Deploy 2 copies of your vm, 1 to a datacentre in zone1, the other vm to the other availability zone

Availability Sets

  • Fault domains (3 by default), ie separate server racks which have separate power etc. Your vm is deployed to say all 3 fault domains and then if a fault domain goes down your still good on the other 2.
  • Update Domains (5 by default), when your vm might need updating, this concept means that some copies can be updated so that others stay up

If you add 6 vm’s to an availability set then the 6th vm would go into update domain 0 as the numbering starts at 0.

Azure Load Balancer (works at layer 4)

  • Is used to distribute traffic to virtual machines
  • Increases fault tolerance and availability for your application
  • Works at the network layer
  • Uses a public Ip address in front of the Azure Load Balancer
  • The back end pool is literally your Virtual Machines
  • The load balancer uses a health probe which needs the protocol, port, interval and threshold set

Important Notes:-

  • The load balancer cannot be used to route traffic between resources in different regions, only the same region.
  • If you want to achieve a higher availability of 99.99% then you should use a Standard Load Balancer instead of a Basic Load Balancer, and have at least 2 healthy virtual machines in the backend pool of the load balancer.
  • The vm’s should be assigned a standard static public IP address

Application Gateway (works at layer 7)

  • Web Traffic Load Balancer
  • Works at the application layer
  • URL Routing – example would be /video goes to backendpool1, /images goes to backendpool2
  • SSL termination

WAF (web application firewall)

  • Centralized protection for your web applications from common exploits and vulnerabilities
  • If you want to deploy an application gateway you need an empty subnet available for your virtual network.
  • SLA 99.5% – 2 or more medium or large instances

Azure Traffic Manager

  • DNS based traffic load balancer
  • Can Distribute traffic across regions
  • You can use different traffic routing methods
    • Priority – choose which region you prefer
    • Geographic – direct end users to specific endpoints based on geographic location
    • Multivalue – all healthy endpoints are returned to the user
  • If your using Azure Site Recovery then you have to create an Azure Site Recovery Vault to store the data
  • Premium Storage tier only allows storage of blobs, nothing else
  • Default NSG Rules – deny all inbound from internet, allow all outbound to the internet, to stop subnets having access out add a new NSG rule and add a service tag of internet, destination port ranges * and then Action Deny with a low priority value of say 100 so that it over rules the default NSG outbound security rules
  • If you want to get access to the Windows Graphic Device interface use Azure Batch
  • When creating an Azure gateway the Ip Address has to be a public static ip address (sku standard)
  • Using Powershell to get an azure keyvault secret
    (Get-AZKeyVaultSecret -vaultname ‘myvaultname’ -name ‘mysecretname’ ).SecretValueText
  • Azure AD Conditional access requires Premium Tier on Azure AD
  • When you set up ASR in another region and point it to some VMs, it installs the Azure Site Recovery extension called Mobility Service in the source VMs
  • Azure Site Recovery is for replicating Virtual or Physical Machines from various sources. It does NOT support Azure App Services. But it does support Hyper-V and VMWare Virtual Machines, and Windows or Linux Physical Machines.
  • ASR requires port 443 and 9443 in order to do it’s replication from the source servers
  • To replicate Hyper-V virtual machines between two on-premises data centers, you need SCVMM to be on both systems already
  • ASR can replicate sites between regions as long as they are in the same geography. It would not support US East machines being replicated to Japan East because it crosses a geographic boundary.
  • VMs across multiple Availability Zones provides the highest Microsoft SLA at 99.99%. Using availability sets provides 99.95% SLA. Standalone VMs behind a load balancer does not provide an SLA. Using Azure Site Recovery provides Business Continuity, and not a high-availability.
  • How does SQL Database implement high availability at the Premium Tier?

The Premium tier of SQL Database runs the database in a 4-node Always On Availability Group Cluster. This has one primary database node with 3 secondary processes keeping copies of the data.

  • Using SQL Database Always On Encryption with Deterministic Encryption. This allows the database to perform database operations on the table such as joins and equality tests, while keeping the data encrypted in the table and from regular application reads. SQL Database Always On Encryption with Randomized Encryption does not allow table operations
  • With Storage Queues, calling UpdateMessage can be used to extend the lease and prevent the message from being given to another process. RenewLock is for Service Bus Queue and not Storage Queues. Rearchitecting the application may not be a simple solution, although it may be wise.
  • You can scale a web app using metrics provided by Application Insights, which needs to be implemented before you can enable such scaling
  • Transparent Data Encryption allows the data stored on the disk to be encrypted and it supports geo-replication and geo-restore. Always Encrypted will not suffice as this is focused on transport encryption (data in transit is encrypted)
  • Azure Confidential Compute (ACC) is only supported on the DC-Series VMs, Azure Confidential Compute allows code and data in the processor to be secured when running. Azure Confidential Compute is not supported on any other VM series except DC-series.
  • SendGrid is an email solution which provides email functionality via distribution groups as well as metric gathering
  • Azure AD Privileged Identity Management is a tool that will allow you to see who has elevated permissions within your environment. You can examine the history of that access, and whether they use those permissions. And you can ask users to justify the need for those elevated permissions in a security review.
  • Azure Site Recovery (ASR) does not support the recovery of most PaaS solutions such as Azure Storage and Azure App Services. ASR is for infrastructure workloads such as Windows and Linux VM’s, SAP, VMWare, Sharepoint, IIS, and SQL Server
  • Function Keys and Azure API Management can both protect a Function app’s public endpoint. Function keys are unique codes that can be required to be used when calling an endpoint. This only protects the endpoint when the function key is a true secret. Azure API Management can be put in front of the function and require other forms of authentication such as Azure AD or OAuth. Functions do not support Shared Access Signatures (SAS).
  • Shared Access Signatures (SAS) and Azure API Management can both protect a Service Bus’ public endpoint. Shared Access Signatures (SAS) are unique codes that can be required to be used when calling an endpoint. This is why they are called “shared”. This only protects the endpoint when the SAS is a true secret. Azure API Management can be put in front of the function and require other forms of authentication such as Azure AD or OAuth. Service Bus does not support Function Keys or Multi-Factor Authentication.
  • Always Encrypt allows you to choose which columns to encrypt, and SQL Database will do the work for you. When using a command line, the data will come out encrypted. But a trusted application can see the data, and use it in JOINs, SELECTs, and WHERE clauses. Application side encryption will not allow JOINs, etc. A Trusted Execution Environment (TEE) is not used for SQL Database service. All data is stored at rest encrypted using TDE by default.


Azure Certification Exams Passed

Proud to say that I now have the following exams passed: –

  • Azure solutions Architect Expert
  • Azure Devops Engineer Expert
  • Azure Developer Associate



Azure Exam Resources

A colleague at work found some amazing resources for Azure exams, I thought it best to share the resources, hope you find them as useful as I have for the exams, please share the link, the courses are all free from EDX. Even if they become invalid the learning content here is fantastic!

MS-100: Microsoft 365 Identity and Services


MS-101: Microsoft 365 Mobility and Security


AZ-100: Microsoft Azure Infrastructure and Deployment


AZ-101: Microsoft Azure Integration and Security


AZ-200: Microsoft Azure Developer Core Solutions


AZ-201: Microsoft Azure Developer Advanced Solutions


AZ-300: Microsoft Azure Architect Technologies


AZ-301: Microsoft Azure Architect Design


AZ-401: Microsoft Azure DevOps Solutions (this is the AZ-400 exam content)


AZ-900: Microsoft Azure Fundamentals


MS-900 Microsoft 365 Fundamentals

Bonus section includes links to the above and more: – https://partner.microsoft.com/en-US/training/assets#/?type=Exam

All the exam learning paths can be found here: –

https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RWtQJJ

Please leave feedback questions etc in the comments section below.



Replacing Azure Automation using Azure CLI and Azure Devops

A customer at work has several Azure Virtual Machines and they wanted to have them stopped between the hours of say 10pm until 6am, nothing too difficult there. I setup an Azure Automation account with a Start/Stop VM Solution, long story short it doesn’t really work, like at all, its messy etc, it’s just a mess.

Note – the solution presented below means zero resource provisioning!

I decided to look at a different way of doing it and asked around and a colleague Nathanat work suggested Azure Functions and also mentioned he had been looking at the Azure CLI of late.

The below is the solution that he came up with, I like this and decided to go ahead and pinch his idea, don’t worry he works in my team and I will give credit where credit is very much due 🙂 – now before I go any further, yes I could use PowerShell, Azure Functions, etc. etc. but I like this implementation mainly because I learned a couple of new things, and if I’m learning then all good.

He wrote 2 bash scripts which invoke the Azure CLI, you pass in the name of the Resource Group(s) and it’ll loops through and starts / stops all the VM’s in your Resource Group(s).

Start VM Bash Script:-


Stop VM Bash Script:-


Start VM Yaml Build Script: –


Stop VM Yaml Build Script: –

Then to ensure the bash scripts run every day you set the schedule for the build and your good to go, like so:-

Feedback welcome, I like this idea for the following reasons: –

  • Learned some YAML
  • Learned about trigger (think scheduled builds) on YAML builds
  • And looked more at the Azure CLI which is of huge interest

Please also follow Nathan on twitter.



Azure Devops Pass Variables between Tasks

Today I wanted to see if it was possible to create a variable in Azure Devops, change the value within a Task and then use the updated value in a step further down the list of Tasks.

 

Turns out its pretty easy (when you get the syntax correct)


So I created a variable in Azure Devops called Version like so and set its value to 1.0

 

 

 

And then I want to make sure I can read this from a standard PowerShell Task in a step within my Build like so :-

Which when built showed me the value as I’d expect of 1.0

And then I want to set the variable to a new value (which could be from anything or anywhere to be honest) using the syntax

##vso[task.setvariable variable=Version]1.2.3

And then finally read out the current value by using $(Version)

Which shows the Version parameter has been updated to 1.2.3 as we would want.

Hope this helps someone at some point 🙂



Azure Devops – Add your build status badges to your Wiki

Its always a good idea on your project to keep your project documentation up to date, I personally like to make use of the Wiki inside Azure Devops, we use Azure Devops almost exclusively at work now.

On the wiki we have a page which documents the Azure Builds and Release pipelines, so that people can get an idea of what the individual builds are for and explain the steps within the Release pipelines, for the most part this is really straightforward, but for new people joining the team it just makes life easier to have this kind of thing written down and explained.

On that note I wanted to show you how to add the status badges for each build to your Wiki, it took me a wee while to find this so I thought I’d blog it because I’ll forget and so other people can see how to do it.

An example of the kind of thing I am talking about is below: –

So how do you find the Markdown for the badges so that you can add this to your wiki or elsewhere?

If you browse to your build(s) for your projects, click on the 3 ellipses on the right hand side, next to the Edit and Queue buttons and then choose Status Badge

Then you need to select the text next to Markdown, and then just paste this into your wiki page.

Hopefully someone finds this useful, bye for now.



Azure Devops – Release Gates

In this blog post I want to talk to you about release gates within Azure Devops, release gates can be useful if you want to add in some further pipeline checks to stop the release going ahead.

Nothing better than an example so here is how to set up gated releases using Azure Devops.


Example

This example shows how you can add in a release gate so that the release wont go ahead and deploy if say there are still open bug tasks within the Azure Board for the current sprint.

Once you have a release, first, click on the lightning bolt on the stage as seen below, and then enable the Gates are on the right hand side.

One this had been selected choose Add and then select Query Work Items, for this I have created a Shared Query where I created a shared query to show me if there are any bugs which are sitting as Approved (which I’m using as open but not started as yet), I don’t want the release to go ahead if there are any bugs in the Approved status.


Note:- In order to create a new query within Azure Devops on the left hand side select Boards, queries and then select new query.

An example query would look something like the following


Fill out the screen below like below and I set the upper threshold to 0.

To recap, I want my release to fail the gate so that the release wont go ahead because I have open bugs within my Azure Board for this particular project.

There are a number of different types of release gates you can use and here is a screen shot of the ones available to use at this time.

I hope you find this useful, if you have any questions please leave feedback.



Podcast appearance on CloudSkills.fm

On March 12th 2019, Mike Pfeiffer invited me to appear on his excellent podcast which is called CloudSkills.fm

CloudSkills.fm is a weekly podcast with technical tips and career advice for people working in the cloud computing industry

Mike Pfeiffer is a Pluralsight author, consultant, advisor, author, and mentor for people ramping up on cloud-based technologies and it was a privilege to have him invite me onto his show and chat about Azure Certifications and also Azure Devops. If you haven’t checked out his podcast as yet, then please do @ cloudskills.fm

In my episode we talk about how I studied for the Azure exams, what content and methods of studying I use and we also talk about how I make use of Azure Devops with my companies customers, where we use it to deploy web apps and infrastructure as code.

I have also been working on moving customer’s data from on premises to Azure using Azure Sql, Azure Data Factory and also using Analysis services with Power BI reporting capabilities, more on that coming soon on my blog. For this project we are building everything from the ground up using Arm templates, deploying the entire resource group to Azure and populating the Azure Data Factory with pipelines all using Azure Devops.

Before I blog about that, lets talk more about my podcast appearance. It’s always a pleasure to be able to talk on podcasts about what I get up to and I like to share what I have been doing with Azure on my blog here. Mike is a very knowledgeable guy and like myself is insanely busy working on multiple things whilst trying to learn and keep up to date, he also has a weekly mailing where he mentions the interesting things he comes across week to week.

I’ll keep this post short and move onto blogging about the Data side of things I’ve been involved in, but if your thinking of doing some Azure exams and wont to know where to get started then have a listen to the episode and give me feedback on it.

Enjoy listening if you take the time to tune in.

 



2018 Retro

I’m not a huge fan of dwelling on past accomplishments, I’m more interested in what’s to come but here goes…

Last year involved the following:

  • Got a new job as Head of Development Services for Sword IT
  • Worked on a large Azure web app with Azure Functions and CosmosDB project straight out the gate
  • Invited onto the board for the Glasgow Azure User Group where I help organise the meetups
  • Spoke at the first Glasgow Azure Global Bootcamp
  • Organising the 2019 Glasgow Azure Global Bootcamp
  • Feature in the top 20 Featured Azure Blogs https://blog.feedspot.com/microsoft_azure_blogs/
  • Sat 8 azure exams (7 betas) within 4 months – I knew zero Azure at the start of 2018.
  • Passed the Architecting Microsoft Azure Solutions (70-535 exam)
  • Wrote 49 blog posts mostly on Azure and Azure Devops
  • First ever Open Source PR accepted (for docs.microsoft.com)
  • Self nominated for MVP
  • Passed 2000 followers on Twitter
  • Technical Reviewer for an Azure related book
  • Started recording YouTube video’s on Azure Devops

Things to look forward to

This year more of the same with a visit to Microsoft Ignite (November) planned and hopefully the Ignite Tour in London late Feb.

 



Azure Devops – OSS Scanning using WhiteSource

Its about time your AzureDevops builds were scanning for OSS vulnerabilities, well your in luck as you can use this Marketplace Extension which is FREE: – https://marketplace.visualstudio.com/azuredevops/

One you add the extension to your Organization you can add it into your build like so:-

This will scan your oss code and give you a detailed report on any vulnerabilities within your Azure Devops repository – #winning.

I have added it to a build I have and here is a sample of the report which you’ll see produced once you’ve added it into the build step.

The report looks like this: –

And below this you’ll see the following: –

An you’ll also see this: –

And this:-

Now you’ll get a report on open source vulnerabilities in your builds 🙂