Category: Azure

1 year in…

Background

Just before covid started I applied for a new job at a company called Intercept in the Netherlands, I had previously only met one of their employees Wesley Haakman whilst at Ignite in Orlando in 2019 I think it was. I spoke to him asking about what they were up to and then spoke to Holly Lehman and asked her opinion on the company, and by this time I was swaying towards accepting the position offered, after I spoke to Holly I was convinced it was the right move for me.

I joined Intercept later that year on September 1st, 2020. 1 year today.

Intercept isn’t like any other company I had worked for before, other companies were good for different reasons, but where I work now is different, more on that later.

During my time here at Intercept I reckon I have worked on at the very least, 26 different projects in the last 12 months using a large variety of Azure services. My background was as a developer, devops, sre and then a development team manager.


Tech

I chose Intercept mainly for my love of Azure and wanting to work on it day to day.

I have worked with the following Azure technologies thus far:-

Azure Networking, Azure Data Factory, Databricks, Event Hubs, Synapse Analytics, App Service, Azure Functions, Container Registry, Virtual Machines, Azure DevOps, Cosmos DB, MySQL, PostgreSQL, Azure SQL, Azure Active Directory, Azure AD B2C, Azure Key Vault, Security Centre, Azure Api Management, Azure API for FIHR, Azure Event Grid, Logic Apps, Noification Hubs, Service Bus, Automation, Azure Backup, Azure Lighthouse, Azure Monitor, Azure Policy, Azure Portal, Cloud Shell, Cost Management, Azure CDN, Communication Services, Azure Migrate, Site Recovery, Application Gateway, Azure Bastion, Azure DNS, Azure Firewall, Azure Front Door, ExpressRoute, Load Balancer, Traffic Manager, VPN Gateway, Azure Storage, Azure Data Lake Storage and more.


Workshops

I have presented 15 or more workshops covering Azure DevOps, Github Action, Azure Fundamentals and Cost Management. During covid we have done them all remotely, cannot wait until we deliver them in person at Microsoft offices around Europe – really looking forward to that.


Culture

The culture at Intercept is simply awesome, you’ll have to take my word for it, I don’t want to go on about it but I thank my lucky stars I work here regularly. We have a lot of fun and have what we call the Intercept Cafe 3 times a week where you can drop in and shoot the breeze with anyone who joins (i love this idea!).

Days off are encouraged, we work flexible hours, everyone speaks English very well (except me of course) and we win as a team, lose as a team, but we mainly kick butt as a team.


Day to Day

Day to day we create designs for customer all around Europe and then we implement the design. We also help customers with improving existing solutions and have teams of people in what we call the continous improvement teams. We only do Azure at Intercept and we work mainly with ISV’s (Independent Software Vendors).


Intercept are Hiring

I would never normally write a blog post about work, and mention were hiring, but things are different at Intercept. I went on holiday and had ZERO stress or feeling in my stomach about returning to work and that was a first.

We have colleagues in the Netherlands, England, Scotland, Northern Ireland, Germany and South Africa and had a recent colleague join from South America who moved to the Netherlands – we chat, play bingo, play xbox, and more.

Take a look at our vacancies page https://intercept.cloud/en/vacancies/


Other Stuff

This summer I decided to down tools and stop studying for exams, stop blogging, stop recording content and just enjoy the summer, thankfully its been the best summer I can remember. I have enjoyed golfing and still doing my bit in the community helping people but havent been sat at my laptop anywhere near as much as I have in the past.

I was renewed as an MCT and MVP and auditioned for being a trainer on LinkedIn Learning. The training part has been something have been thinking about doing for a while now, finding the time was impossible but not I think I may have decided thats the way I want to go.

I realise I love helping people, and whatever that entails, helping people get a leg up, bevoming MVP’s, helping people by mentoring them etc is what I love doing.


Career

People still ask me what I want to do, do I have any career goals? – I can honestly say at this moment I am enjoying life at Intercept. I love working with Azure, I love learning hence why I do so many exams, I have a passion for learning and a passion for helping people which will never leave me. If you’re not currently learning something you’re going backwards.

Don’t forget to subscribe to my YouTube Channel.



How to set environment variables for use with an Azure Function

There are many ways to pass variables into an Azure Function.

In this quick blog post I will show you how you can test the Azure Function locally with local settings and then use app settings from the Azure Portal and then also use values stored within Azure KeyVault incase we need to store and retrive secrets.

Ok, so to run our Azure Function locally I prefer to use C# Azure Functions, they just work, I can debug them in VS Code or Visual Studio – I am a dinosaur and been using Visual Studio since it first came out so I tend to stick with using that over VS Code, yes I am that old.

Moving on lets show how we can make use of variables whilst debugging locally, for this we just need to create a local.settings.json file like the one below:-

{
  "IsEncrypted": false,
  "Values": {
    "FUNCTIONS_WORKER_RUNTIME": "dotnet",
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "DemoUsername": "azuregreg",
    "DemoPassword": "letmein"
  }
}

So our code can now use a stored username and password to test with locally – we don’t check this file in as its only for testing locally (add a .gitignore file).

Ok so we can test our Azure Function with code that accesses the local variable and away we go.

So our function now outputs the following (remember this is from local settings) :-

this is gregors demo function - the username is azuregreg and the password is letmein


Now the interesting part next is how do we store variables in Azure and make use of them.

So, we have some options available to us, for things like username and password we can store them in Azure KeyVault, if they are simple settings then we can store them in appsettings within the Azure Portal.

Let’s take a look at storing these variable in the App Settings section of the Azure Portal for our Azure Functions.

In the screen shot above we are in the Azure Portal and clicked into our Azure Function app and then click on Configuration and then + New application setting

Now we can add configuration values to our code, we can store setting here but sometimes we need to store secrets and for this we use Azure KeyVault, which we will return to shortly.

So lets add the username and password settings into the application settings and see how we can use them first of all.

So I went ahead and added them into the application settings section like so:-

In the screen shot above I clicked Show Values so that I can show you, the reader, the values I set exactly the same properties in the local.settings.json file when debugging locally – click Save.

Ok, so now we have added in some application settings, how does the code need to change to pick up these values from this area in the Portal? – well the good answer is we don’t need to change our code, it works exactly the same way as whilst using the local.settings.json file.

So the above works, but I hate when people say yeah but it works, yeah it works but can we make it better should always be the question – getting things working and making it as secure as it can be are different things altogether.

So why would we want to perhaps store the username and password in KeyVault? – glad you asked.

Maybe there is a username and password which you don’t want everyone to know, so you can add these values to KeyVault and not give anyone access to read the values for the password for example, but your application can read and use the value from the keyvault without anyone being able to see the password in plain text – sounds good to me so lets go set this up.

If you want to test this out I created a C# Azure Function which is basically just the following code, reminder the local.settings.json is at the start of the blog post.

C# Azure function code below.

[FunctionName("Function1")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req, ILogger log)
{
    log.LogInformation("C# HTTP trigger function processed a request.");

    string username = Environment.GetEnvironmentVariable("DemoUsername");
    string password = Environment.GetEnvironmentVariable("DemoPassword");

    string responseMessage = "this is gregors demo function";

    return new OkObjectResult(responseMessage);
}

First we need to create a KeyVault and one you’ve done that, locate it and click on Secrets and create 2 new secrets, lets call then username and password for simplicity – set the values like so:-

username: azuregregviaappsettings
password: letmeinviaappsettings

When we run our Azure HTTP Function this time it picks up the values from the appsettings we just updated in the app settings section of the Azure function in the portal.

So our function now outputs:-

this is gregors demo function - the username is azuregregviaappsettings and the password is letmeinviaappsettings

Lets now change our existing app settings so that we get the details from KeyVault rather than just simply storing them in our app settings (where anyone can see them) – only people with the correct RBAC user rights can see our KeyVault secrets.

Ok now that we have our 2 secrets lets try to access them from the appsettings section of our Azure Function. Go back to the Azure Function and then click on Configuration and then we will be back at the screen that shows us our current applications settings.

Now lets edit the existing app setting called DemoUsername by finding the row and selecting the edit button as below :-

Lets past in the following
@Microsoft.KeyVault(SecretUri=https://<your keyvault name>.vault.azure.net/secrets/username) replacing <your keyvault name> with the name of your KeyVault like below.

Once you’ve done this for the username and password we can now use the values from our Azure Function.

Before this works we need to do a couple more steps – we need to create a managed identity and then also create an access policy within KeyVault.

To create a managed identity go to your Azure Function and then under Settings, select Identity. Change the status to On and click Save, also take a copy of the Object ID as we will need this later on.

Next we need to create an access policy within Kay Vault, so go into you’re KeyVault and select Access Policies, and then choose the + Add Access Policy link. Where it says Select principal, click the words none selected and then paste in the Object ID we took a note of above and then select it. Then within the Secret permission drop down select Get and List and then click Add.

Now go back to your Azure Function and select Configuration and then edit both your DemoUsername and DemoPassword app settings and then click save, They should now look like this:-

Notice in the screen shot above the Green Tick next to the words Key vault Reference, if this is red then check your steps with the managed identity, and creating the access policy above.

So our function now outputs:- 

this is gregors demo function - the username is azuregregfromkeyvault and the password is letmeinviakeyvault

And that is how you can run and test or function locally, using local.settings.json but also stored environment variables in app settings but also store them in KeyVault if they are secrets.

Don’t forget to subscribe to my YouTube Channel.



AI-102 Azure Study Guide

In this blog post I cover all of the resources I came across whilst studying for the AZ-102 Exam

AI-102: AI Engineer on GitHub: – https://github.com/MicrosoftLearning/AI-102-AIEngineerhttps://github.com/MicrosoftLearning/AI-102-AIEngineer

And this too:-

https://microsoftlearning.github.io/AI-102-AIEngineer/

Plan and Manage an Azure Cognitive Services Solution (15-20%)
Select the appropriate Cognitive Services resource



Azure DP-300

Azure DP-300 Exam Study guide

Plan and Implement Data Platform Resources (15-20%)
Deploy resources by using manual methods



Azure App Service

Troubleshooting App Services in Azure

In this blog post, I wanted to cover how to go about troubleshooting an App Service in Aure which is a web app with a SQL server backend whereby users have reported issues with the slow performance of the website.

The first thinh I tend to look at is the backend store, in this case, Azure SQL Server and we have some really great tooling we can use to troubleshoot perforamce issues with Azure SQL.

The first port of call was to open up the Azure Portal and go to the Resource Group with the issues and click on the SQL Server database and head to the Intelligent Performance section on the left-hand menu as highlight below: –

Performance Overview
This currently has a recommendations area that suggests adding 5 different Indexes which are all set as HIGH impact.

Indexes can sometimes cause adverse effects so it’s recommended to look at the suggestions, copy the script from the recommendations and consider if this Index will indeed help with the performance of queries.

Query Performance Insight
The second area I look at is query performance insight and from here we can see the average CPU, Data IO, Log IO on the SQL Server database across the last 24 Hours as an average. We also get an insight into what queries are running and taking the longest time to complete.

I changed the graph above to show the last 7 days and I can see CPU is maxed out at 100% for a long period within the last 7 days as seen below:-

Long Running Queries
This area identifies queries which are taking a long time to complete and always worth checking on this regularly.
The following is a screen shot of long running queries within the database for the past week. To find this information select the database instance in the portal and then select Query Performance Insight and select Long running queries, then I chose custom and changed the time period to Past week.

We can see above the yellow query is the database query which has the longest duration this past week, you can click on the yellow area and it will show you the details of the query which is a long running query.

Automatic Tuning

Azure SQL Database built-in intelligence automatically tunes your databases to optimize performance. What can automatic tuning do for you?

  • Automated performance tuning of databases
  • Automated verification of performance gains
  • Automated rollback and self-correction
  • Tuning history
  • Tuning action Transact-SQL (T-SQL) scripts for manual deployments
  • Proactive workload performance monitoring
  • Scale out capability on hundreds of thousands of databases
  • Positive impact to DevOps resources and the total cost of ownership

I would recommend that FVN turn this on and leave it like the following:-

This means that Azure will tune the indexes using built in intelligence and create indexes when it thinks you need them based on usage patterns. A word of caution here as these recommendations aren’t always correct so please bare this in mind.

Log Analytics
I always recommend adding the Azure SQL Analytics workspace solution to the subscription and this gives us further insight into the SQL Server in Azure. Once you turn this on you need to wait sometime before it can gather a decent amount of data.

The screen shot below shows us the type of information we can get from it, this screen shot was taken not long after being turned on so if you wait some time it will have much more useful details:-

From here we can get more information about deadlocks, timeouts, etc.


Now lets take a look at the website which is in an App Service in Azure and see what tool we can use to help us troubleshoot issues with the performance.

I always recommned adding Application Insights into Azure for resources when possible, and here if we click on the App Insights for the web app we can instantly get soe basic info. If you click on the Application Dashboard as seen below we get a high level vue of whats going on in our App Service.

The Application dashboard for a typical web app might look something like this: –

Ok, so let’s now do some further investigation into our app service issues. This time I chose the App Service itself and then I chose Diagnose and solve problems from the left-hand menu. This feature is underused in my opinion and is very useful indeed, not sure if many people have looked at it but it can be pretty helpful with recommendations and also pointing out some things that you may want to think about remediating.

Once in the Diagnose and solve problems area I usually click on Availability and Performance within the Troubleshooting categories section and if you do, you’ll see something like this: –

In the image above we can see that we have some App Performance issues to go and investigate. Clicking into the App Performance section we get in-depth details about the Performance and we get Observations that say things like Slow Request Execution with details of the web page, average latency, total execution time, etc. The detail here is very helpful in tracking down potential issues in the code, or the configuration of your web application. There are a number of options to check within each section of the 6 troubleshooting categories, an example is shown below for the Availbility and Performance section: –

Summary
In summary, there are a number of really awesome tools to aid us with troubleshooting App Service perormance issues, go check them out the next time your web app is running poorly.



Azure Functions

Azure Durable Functions – Support Caller

I wrote an Azure Durable function which makes a phone call to out of hours support engineers when an alert is raised within their production Azure environment, and I wanted to talk about how I did it and what I used.

When an alert is raised with the customers Azure environment I send an HTTP Post to my Azure durable function endpoint from the reporting tool we use, which is PRTG, you can do the same from Azure just as easily, we use PRTG to monitor Azure resources for things like High CPU and the amount of free disk space remaining, etc.

Durable functions was chosen so that I can make use of what’s called an orchestration durable function – you can read more about durable functions: – https://docs.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-overview?tabs=csharp

If you read the above articles you’ll get a good grasp of what an orchestrator in durable functions can do, to convey why I used them I have the following workflow requirements:-

  1. Receive details of the alert.
  2. Retrieve the support people’s phone numbers.
  3. If an alert is raised call the first number 3 times in 5 minutes, if answered by a human, read out the alert message and some extra content and ask the user to acknowledge the issue by pressing 1 on the keypad.
  4. If the support engineer doesn’t answer after 3 attempts then move onto the next number.
  5. If the support engineer answers and presses 1 stop the orchestration.

1 – Receive details of the alert
This is really easy to do, here I have a template set up in PRTG which forwards the details of the alert to my durable function like so:-

2 – Retrieve the support people’s phone numbers
I am storing the support people’s phone numbers in a CSV file which is uploaded to a simple Azure storage account, this allows the customer to edit the support numbers easily.

3 – Making the call
Here I make use of Twilio Rest API and I create a CallResource object and then call the Create method, Twilio has a thing called Twiml which you can create a message of your own out of and it will read this message out to the person who picks up the phone call. All of the details about who you call, what the call says, and the action they need to take are stored in config so it can be very easily changed for different customers.

The code to make a call is actually really simple.

var call = CallResource.Create(twiml: 
new Twilio.Types.Twiml($"<Response><Gather action='{callbackhandlerURL}'>{messageToReadToUser}"),
to: to,
from: from);

4 / 5 – Answering the call
This was the tricky part, figuring out if they had picked up the call was my initial challenge and I tried numerous things from the Twilio docs which were misleading, didn’t seem to work as I expected. The samples for this part of the documentation are sadly lacking.
Now when the call comes in the support engineer is asked to press 1 to acknowledge they have received the call and the orchestration can end, part of this involves having a callback URL so that Twilio can send you details back to a URL of your choice so that you can get the details of the call, things like call length, etc and if they pressed 1 during the call.


Orchestration
The orchestration part was pretty tricky for me to get right, huge thanks to @marcduiker he was an enormous help to me on this, figuring out how to do some of the steps proved tricky but very interesting!

Marc is putting together an Azure Functions University series where you can go and learn all about Azure Functions – please go check that out.


The orchestration logic was something like the following:-

MainOrchestrator – this function’s job is to be the orchestrator, within this function we call sub orchestrators, and also activity functions, think of an activity function as a separate function that does something, I had a GetNumbersFromStorage activity function and a SendNotification activity function. so the idea behind durable functions is to be able to call multiple azure functions using patterns, one of which is the orchestrator pattern.

RetryOrchestrator – this function’s job is to work out what to do when the call wasn’t answered the first time, do we need to make another call, how many times have we called this number, and have we ensured that the calls are spread out of 5 minutes so we don’t make multiple calls at the same time.


Twilio
To make this all work I created a Twilio account and purchased a number, this means you can use this number to make the calls. It costs 2 pence per call and 7 pence per call if you want to detect if someone answered the call using answering machine detection, so there are options available.

Summary
Durable functions have a lot of great use cases, definetly check them out and build something yourself to get a handle on how they work. The Azure durable function docs are really good.




AKS Zero To Hero – Series for everyone

Richard Hooper and I have started a new series called AKS Zero to Hero, the aim here is for Richard to teach me AKS from zero to knowledge to hopefully becoming a hero when it comes to AKS.

We see a lot of customers either already using AKS or wanting help getting started with AKS so it’s about time I got up to speed. If you are new to AKS or a seasoned professional we will be covering as much AKS content as we possibly can, the aim is to try to have content out each week.

We will be taking an asp.net core project which will be open-sourced on GitHub at https://github.com/CloudFamily/AKS_Zero_to_Hero and we will deploy this to AKS and cover as many areas of AKS as we can possibly cover. The series will run for a while so please hit subscribe and click on the bell notification to be alerted when a new video drops.

The YouTube playlist for all of our videos thus far can be found below.

Please give us feedback, ask questions etc and we can try and answer them in an ask me anything session which we will be planning within the next month.

Don’t forget to subscribe to my own YouTube Channel.



Using the latest version of the Azure CLI

In this blog post, I wanted to quickly cover how you can keep the Azure CLI up to date on your local system and within Azure. I use the Azure CLI as my go-to choice for writing deployment scripts in Azure. The reason you want to keep this up to date is for new additions as we all bug fixes for previous versions.

The Azure command-line interface (Azure CLI) is a set of commands used to create and manage Azure resources. The Azure CLI is available across Azure services and is designed to get you working quickly with Azure, with an emphasis on automation.

Its super simple to keep this up to date and you can do this by opening a PowerShell or Bash script window and typing:-

az upgrade

But instead of doing this maybe you want to keep it up to date without having to keep checking, you can also do this by using the following command:-

az config set auto-upgrade.enable=yes

But even better yet you can keep the Azure CLI up to date without ever being prompted by using the following command:-

az config set auto-upgrade.prompt=no

And that’s it, no you no longer need to worry about am I using the latest version of the Azure CLI.

You can read more on this at the following URL: – https://docs.microsoft.com/en-us/cli/azure/update-azure-cli?WT.mc_id=AZ-MVP-5003451

Don’t forget to subscribe to my YouTube Channel.



Azure Logic App Api call save a file to Blob Storage

I wanted to see how easy it would be to create a Logic App to call an API and return data from it and then translate the contents into a CSV file, any excuse to learn something new and play with Logic Apps, which is not something I have done a lot of, to be honest.

So my goal was to pick an api, call it using a GET request, grab the json from the Api and then convert this to a csv file and then create a file on blob storage. In this blog post I will show you how I went about it.

I want my Logic App to do this each month and grab the data from the api and create a new blob so lets take a look at the end result and go through it step by step.

So we have 5 steps to accomplish this task.

Recurrence – this is just going to run the logic app on a schedule, so I am running this once each month.
HTTP – here is where I give it the API URL which in my case is https://geocatalogus.nl/api/3/action/datastore_search?resource_id=ecbe6732-5a6b-4858-84db-b03c410ff7aa
and I set the Request type to GET.
Parse JSON – Here I grab the JSON response from the HTTP step above and then parse it by supplying an example of the body from the JSON returned in the HTTP step. This looks like so:-

The Body (green part in the screen shot) is taken from the Dynamic Content where I just typed Body and then clicked on it.
Create CSV Table – now I want to interrogate the Parsed JSON from above and find the part of the JSON I am interested in and for this API I want the part called record which is the data I am interested in.


Again I clicked on the From part above and chose Dynamic content which lists the parts of the JSON returned and from there I chose records, I left the Columns as Automatic and thats all I needed to do here.
Create Blob – Now I want to create a new blob in Azure Storage soI chose that for my last step and gave it the connection details to my Azure Storage Blob container like so:-

I run the Logic App and it calls the API within the HTTP step, parses the returned JSON from the API, I then use the Create CSV Table step to format the data and then save the output from that step by using a Create Blob step.

And that is all there is to it, I did this just to learn something new and remind myself how cool Logic Apps are and how easy they are to use.

Don’t forget to subscribe to my YouTube Channel.



Immutable storage for Azure Storage Blobs

If you have storage blobs containing things like backups or files then Azure now has Immutable storage available for Azure Storage Blobs generally available in all public regions.

Immutable means that it is unable to change or be changed and this means that if a customer has let’s say a backup then they can store this unchanged which for some companies is very nice to have.

To take advantage or to test out immutable storage lets go through what we need to do to test it out.

  • First of all, create a storage account.
  • Click on Containers and create a new container, give it a name and choose Private (no anonymous access).
  • Once created click on the name of your new container and then upload some files.
  • Once you have uploaded some files click on Access Policy on the left-hand side, notice we have 2 sections, Stored access policies and Immutable blob storage, under Immutable blob storage, select Add policy.
  • We now have 2 options to choose from
    • Time-based retention
    • Legal hold

Time-based retention allows us to add a number of days value between 1 day and 400 years, this also makes the files immutable.

Note:- You cannot change this value to 0 at any time. Once the interval you add expires – Upon the expiration of the retention interval, the data will continue to be in a non-modifiable state but can be deleted. Retention policy changes may require some time to take effect. 5 edits are permitted to the policy.

Legal hold retention means you add a tag to the blob container – each legal hold policy needs to be associated with 1 or more tags. Tags are used as a name identifier, such as a case ID, to categorize and view records.

You cannot delete or modify any files with the container whilst there is either a Time-based retention policy or a Legal hold policy, however if you delete the legal hold policy you can then delete or modify files with the container.
With Time-based retention, you can allow additionally protected appends and change the retention interval.
Time-based retentions need to be locked in order to be active and to add a lock click on the 3 dots and choose Lock policy.

Note:- Once you apply the lock you cannot delete the lock and just before you click save on applying the lock you will see the following reminder:-

Summary
I can see some people having the need to keep backups and have them immutable for a number of legal reasons and this new feature will be very handy for them.

Don’t forget to subscribe to my YouTube Channel.