Learning so much! I try to do blog posts that have 5 questions I’m struggling with the amount of research that goes into them being more voluminous than what I’m used to. The very first question required almost 1k words and I had to split the 5 question lot into 2 posts. Thus the confusion on the posting number because you know, I like to stick with an established method of doing things. Very traditional haha… anyway, here is the first question:
I have some confusion about data storage terms and now might be a good time to clear those up. The other part of the question is what kind of data is the message file. Like is it CSV? I would assume it to be as that would most likely be the most efficient way. I’m not sure if I’ll get an answer to that but maybe I’ll learn something by searching for an answer to that question. Regardless, it seems that would plausibly go into a table if that where the case but I can see two problems with that for one, why use SQL for everything and two it might end being fragmented data based on how the conversations are handled. Data lake and blob seem the same to me so lets figure out what that means Azure Data Lake vs Azure Blob Storage in Data Warehousing
I think blob storage is good at non-text based files – database backups, photos, videos and audio files. Whereas data lake I feel is a bit better at large volumes of text data. More often than not, personally, I would choose Data Lake Store if I’m using text file data to be loaded into my data warehouse. Of course, you can use blob storage, but I feel that is for those non-text data that I mentioned above.
Welp, thats helpful. It seems more than ‘opinion based’ but lets find out what MSFT has to say Comparing Azure Data Lake Storage Gen1 and Azure Blob Storage
Based on this, I like the idea that datalake is for text files and worst case scenario we can for sure assume that chat message logs are text based but I’m not sure how they are indexed, outside of using SQL, to find key words and so forth and msft isn’t really saying. It cant be that hard to figure out if you actually have one or look at someones chat data.
The next interesting thing in this is ‘a file share in an Azure storage account’ is this blob storage? haha now stop me if I’m wrong on the differences between hierarchical and folder storage models not being ‘containers’
Regardless, there are more than blob and data lake storage. The confusing part is that no one exactly lays out data lake in articles. I read through several and found this one helpful Microsoft Azure Storage Overview
Azure blob storage: It is optimized to store huge unstructured data. Storage is in terms of binary large objects (BLOBs).
Azure table storage: It has now become a part of Azure Cosmo DB. Azure table stores structured NoSQL data.
Azure file storage: It is a fully managed file sharing service in the cloud or on-premise via the Server Message Block (SMB) protocol.
Azure queue storage: It is a storage service that stores messages that can be accessed through HTTP or HTTPS from any part of the globe.
Disk storage: It is a virtual hard disk (VHD) which is of two types: managed and unmanaged.
Which matches up with what MSFT says in this Introduction to the core Azure Storage services
The Azure Storage platform includes the following data services:
Azure Blobs: A massively scalable object store for text and binary data. Also includes support for big data analytics through Data Lake Storage Gen2.
Azure Files: Managed file shares for cloud or on-premises deployments.
Azure Queues: A messaging store for reliable messaging between application components.
Azure Tables: A NoSQL store for schemaless storage of structured data.
Azure Disks: Block-level storage volumes for Azure VMs.
As you can see it doesn’t mention Data Lake storage but there is a separate article for that Introduction to Azure Data Lake Storage Gen2
Azure Data Lake Storage Gen2 is a set of capabilities dedicated to big data analytics, built on Azure Blob storage. Data Lake Storage Gen2 is the result of converging the capabilities of our two existing storage services, Azure Blob storage and Azure Data Lake Storage Gen1. Features from Azure Data Lake Storage Gen1, such as file system semantics, directory, and file level security and scale are combined with low-cost, tiered storage, high availability/disaster recovery capabilities from Azure Blob storage.
And this leads me to wonder if there will be variations between 1 and 2 noted on the test. I guess we will get to that when it shows up. Anyway, its for big data that you run analytics against. Somehow. Anyway, based on this information I think that the Data lake or data lake answer makes sense.
I’m not sure what these rolls are so lets find that out: What is role-based access control (RBAC) for Azure resources?
Owner – Has full access to all resources including the right to delegate access to others.
Contributor – Can create and manage all types of Azure resources but can’t grant access to others.
Reader – Can view existing Azure resources.
User Access Administrator – Lets you manage user access to Azure resources.
Ok, so this one is pretty straight forward
Haha, this is great, I get to quote my self! And no, I didnt do this on purpose.
Quickly realizing that you gave an answer to a question in the last blog post and then looking at the question and somehow not knowing right away that the answer is what you said in the last blog post, regardless of what you where thinking when going through the questions, is a good sign that this may take longer than expected. Sigh. I’ve also realized that keeping text from articles in a standardized format is more annoying with this method of blogging than using a text editor so I’ll go back and clean that one up. This next one has a lot of screen shots and does not appear to be answerable but I would like to cover the material none the less.
And if you let them sit a few days haha magic…anyway, here is the list of articles possibly related to this topic thus ensuring me there is an overwhelming amount of info covered on this bad boy
Anyway, this is the one we are covering in this question: Configure a VNet-to-VNet VPN gateway connection by using the Azure portal
Wow, that is a long article but the following is what they are talking about with the gateway:
To create a virtual network gateway
- From the Azure portal menu, select Create a resource.
- In the Search the Marketplace field, type ‘Virtual Network Gateway’. Locate Virtual network gateway in the search return and select the entry. On the Virtual network gateway page, select Create. This opens the Create virtual network gateway page.
- On the Basics tab, fill in the values for your virtual network gateway.
Project details- Subscription: Select the subscription you want to use from the dropdown.
- Resource Group: This setting is autofilled when you select your virtual network on this page.
- Name: Name your gateway. Naming your gateway not the same as naming a gateway subnet. It’s the name of the gateway object you are creating.
- Region: Select the region in which you want to create this resource. The region for the gateway must be the same as the virtual network.
- Gateway type: Select VPN. VPN gateways use the virtual network gateway type VPN.
- VPN type: Select the VPN type that is specified for your configuration. Most configurations require a Route-based VPN type.
- SKU: Select the gateway SKU from the dropdown. The SKUs listed in the dropdown depend on the VPN type you select. For more information about gateway SKUs, see Gateway SKUs.
- Generation: For information about VPN Gateway Generation, see Gateway SKUs.
- Virtual network: From the dropdown, select the virtual network to which you want to add this gateway.
- Gateway subnet address range: This field only appears if your VNet doesn’t have a gateway subnet. If possible, make the range /27 or larger (/26,/25 etc.). We don’t recommend creating a range any smaller than /28. If you already have a gateway subnet, you can view GatewaySubnet details by navigating to your virtual network. Click Subnets to view the range. If you want to change the range, you can delete and recreate the GatewaySubnet.
The Gateway subnet range seems confusing until you assume they are using NAT and on the other side of the Gateway it wants to know what range of addresses are going through that gateway on the VLAN. Why? Honestly, no clue. Which adds to the confusion of ‘am I thinking correctly about why it asks for this’ normally it should ask for a gateway when going between VLANS. Assuming thats what a VNET is? Maybe I should verify that too: Azure Virtual Network frequently asked questions (FAQ)
What is an Azure Virtual Network (VNet)?
An Azure Virtual Network (VNet) is a representation of your own network in the cloud. It is a logical isolation of the Azure cloud dedicated to your subscription. You can use VNets to provision and manage virtual private networks (VPNs) in Azure and, optionally, link the VNets with other VNets in Azure, or with your on-premises IT infrastructure to create hybrid or cross-premises solutions. Each VNet you create has its own CIDR block and can be linked to other VNets and on-premises networks as long as the CIDR blocks do not overlap. You also have control of DNS server settings for VNets, and segmentation of the VNet into subnets.
Use VNets to:
Create a dedicated private cloud-only VNet. Sometimes you don’t require a cross-premises configuration for your solution. When you create a VNet, your services and VMs within your VNet can communicate directly and securely with each other in the cloud. You can still configure endpoint connections for the VMs and services that require Internet communication, as part of your solution.
Securely extend your data center. With VNets, you can build traditional site-to-site (S2S) VPNs to securely scale your datacenter capacity. S2S VPNs use IPSEC to provide a secure connection between your corporate VPN gateway and Azure.
Enable hybrid cloud scenarios. VNets give you the flexibility to support a range of hybrid cloud scenarios. You can securely connect cloud-based applications to any type of on-premises system such as mainframes and Unix systems
So basically its the Azure version of a VLAN. Ok, so this block editor is super buggy and bear with me if there is some formatting issues. Using the block editor is a nightmare if you bring over test with both a header and a ordered list. It splits them into separate blocks that it wont merge, one will allow you to edit the text into a quote and the other wont. Its kind of a pain in the ass. You have to switch it to HTML and then delete the second block and be careful you select the right block because I accidentally deleted this paragraph and then it wouldn’t let me use the back button to restore it. So I’m learning as I go with this as opposed to stressing over formatting too much. Seems like a waste of time to become a quick expert.
Now I’ve got this block quote text that I cant get to go away lol, fun. Anyway, I feel like I covered the topics at hand and will try and go back and adjust formatting on the previous post at some point. Humm, I think I resolved that by changing the blockquote HTLM that auto populates into a paragraph and then it didnt like that so I switched it over to a classic block and removed it. That seems to have worked…thanks for all your help in illuminating these issues. Good wrk
Anyway, there was an additional question in this lot but its basically another, post unto its self type of question and Ill get back to it later. That’s all for now!
xAFTERHOURSx 
Leave a Reply