I hadn’t considered this but yeah those licenses are expensive.
If you already have existing Windows Server and SQL Server on-premises licenses with Software Assurance, you can use them for Azure virtual machines (VMs). This will allow you to save the Pay-as-you-go cost for Windows Server and SQL Server licenses. The Azure Hybrid Benefit applies not only to Azure VMs but also on Azure SQL Database PaaS services and the Azure Dedicated Host. If you want to know more about how to take advantage of the Azure Hybrid Benefit, check out the Microsoft Azure Docs page.