Connecting Azure services to Vantage

Connect Microsoft Azure Services to Vantage

2022年3月11日 13 最小阅读

Many Teradata customers have interest in connecting Teradata Vantage as a Service with Microsoft Azure First Party Services. This guide describes ‘high-level’ Vantage as a Service connectivity options with the Azure Services.

Although this approach has been implemented and tested internally by Teradata, it is offered on an as-is basis. Though, Azure connectivity options have been tested with Vantage as a Service, neither Microsoft nor Teradata provide validation given customer architecture and security requirements can vary greatly. Hence, guide should be viewed as a high-level reference only.

This guide includes content from both Microsoft Azure and Teradata product documentation.

Overview

This article describes how to connect various Azure Services to Vantage as a Service. You will need to first connect customer’s and Vantage as a Service virtual networks, then install Azure Services gateway and other components on Windows VM in the customer’s virtual network. Finally, depending on the service, create a private endpoint for the service to reach the gateway or customers virtual network to access Vantage as a Service.

Vantage as a Service on Azure supports multiple connectivity options depending on where the customer’s connections originate and the type and number of applications that need connectivity. 

This diagram displays the connectivity options.
 

Azure-services-_1.png

  • VNet Peering – VNet Peering is the preferred option when a Vantage instance in the Teradata VNet needs to initiate connections to multiple entities in the customer’s VNet (e.g., Azure Services offering).
  • Azure Private Link – This is the preferred connectivity method when accessing Vantage from a customer VNet for installations that do not need connection initiation from the Teradata VNet into the customer’s VNet for any application access. Private Link also simplifies IP address planning, has higher network speeds, and keeps traffic within the Azure network. One Private Link connection is included in the price of a Vantage on Azure subscription.
  • Azure ExpressRoute (not covered) – This is the preferred connectivity option when accessing Vantage from on-premises.
  • VPN (not covered) – This is an alternate connectivity option when accessing Vantage from on-premises.

Note: We will cover VNet Peering and Private Link to access Vantage as a Service (SQLE) instance. Other connectivity and components (i.e. ViewPoint, QueryGrid Manager, etc) are not covered.

About Azure Services

The Azure cloud platform is more than 200 products and cloud services designed to help you bring new solutions to life—to solve today’s challenges and create the future. Build, run, and manage applications across multiple clouds, on-premises, and at the edge, with the tools and frameworks of your choice.

Many of the Azure Services support Teradata Vantage using a connector developed by Microsoft.  Azure Data Factory, Logic Apps, Power Apps, Power Automate and Power BI Service just to name a few. These Azure Services uses a Microsoft gateway component to access Teradata Vantage in the cloud and on-premise. 

For more information see documentation.

About Teradata Vantage as a Service

Teradata Vantage™ is our flagship analytic platform offering, which evolved from our industry-leading Teradata® Database. Until references in content are updated to reflect this change, the term Teradata Database is synonymous with Teradata Vantage.

Teradata Vantage is a data analytics platform for performing advanced analytics in the cloud or on-premises. With Vantage you can integrate analytic tools, languages, and engines to get insights from all your data.

With Vantage delivered as a service, Teradata manages the performance, security, availability, and operations of the Vantage platform as described in this service description.
Vantage on Azure

Deploying Vantage on Azure establishes a Vantage environment in a Teradata-owned Azure subscription. Customers can often subscribe to a Vantage environment in the same Azure region as their data.

With Vantage on Azure, customers can access the same software capabilities provided in an on-premises Vantage system but in an Azure environment.

Teradata provisions, configures, and provides customer access to an Azure environment in a supported region.

For more information see documentation. 


Prerequisites

You are expected to be familiar with the Azure networking concepts (VNet Peering, Private Link Services and Private Endpoints), Azure services and Vantage with the following accounts and systems.

  • An Azure account
  • A Teradata Vantage as a Service instance (version 17.0) or later

Procedure

Once you have met the prerequisites, choose virtual network connectivity option:

Virtual Network connectivity

  1. Option 1: VNet Peering connectivity (recommended) 
  2. Option 2: Private Link connectivity

Azure Services connectivity

  1. Create Windows (VM) virtual machine
  2. Install an Azure Service gateway and/or Teradata client software
  3. Create Private Endpoint or requirements for an Azure Service
    1.  Azure Data Factory
    2. Azure Data Factory Managed Virtual Network
    3. Azure Blob Storage
    4. Azure Batch Service

Virtual Network connectivity

Customer and Vantage as a Service virtual networks need to be connected to access Vantage database. Network connectivity is something that’s done prior to handing over the system to a customer. Customers would decide what network connectivity they want during architecture discussions. After the system is provisioned, Cloud Ops team would set up the network connectivity based on the approved architecture of the customer.

If you do not have a virtual network, follow the steps in create virtual network (steps 1-6).

Ensure customer virtual network IP address range and subnet does not overlap with Vantage as a Service

Option 1 - VNet Peering connectivity (recommended)

Azure Virtual Network (VNet) Peering enables you to seamlessly connect Azure virtual networks. Once peered, the VNets appear as one, for connectivity purposes. The traffic between virtual machines in the peered virtual networks is routed through the Microsoft backbone infrastructure, much like traffic is routed between virtual machines in the same VNet, through private IP addresses only. No public internet is involved. You can peer VNets across Azure regions, too.

This diagram displays the VNet Peering connectivity

Azure-services_2.png
Figure 1: VNet Peering connectivity

The following steps are:

Step 1 – Acquire and add ‘shared’ service principal and assign network contributor role

Logon to the Azure portal with Azure Active Directory owner access. 

Browse to your VNet, and then click Access control (IAM) and assign Network Contributor role to the shared service principal provided by the cloud team (i.e. ICAzure_Peering_SP_Prod)

Click Save.
Azure-services_3.png

Figure 2: Virtual network Access control (IAM) role assignment  

Customer can remove service principal after automation script completes VNet Peering.

Step 2 – Provide Tenant/Directory ID and VNet Resource ID to cloud team

Customer can find Tenant/Directory ID by clicking on Azure Active Directory --> Properties 
Azure-services_4.png

Figure 3: Azure Active Directory properties 

VNet Resource ID can found by clicking on Virtual Network --> Properties 

Azure-services_5.png

Figure 4: Virtual network properties   

Step 3 – Cloud team runs automation script to complete VNet Peering

Once automation script completes, click on Peerings under your Virtual Network and check status for Connected

Azure-services_6.png

Figure 5: Virtual network Peerings connections   

Step 4 – Use Vantage as a Service provided IP to access Vantage

Once virtual networks are peered, you can start using Vantage as a Service provided IP addresses.

Step 5 – Remove service principal from Azure Active Directory tenant

Browse to your VNet, click Access control (IAM) and then Role assignments.

Choose the shared service principal provided by the cloud team (i.e. ICAzure_Peering_SP_Prod).

Click Remove

Next, go to Azure Services connectivity section.

Option 2: Private Link connectivity

Azure Private Link service is the reference of a service or resource that is powered by Azure Private Link. In this case, the resource is Vantage as a Service running behind a Azure Standard Load Balancer can be enabled for Private Link access so that consumers can access it privately from their own VNets. Customers can create a private endpoint inside their VNet and map it to this service effectively bringing the Vantage as a Service resource into their VNet.

This diagram displays the Private Link connectivity
Azure-services_7.png

Similar to VNet Peering, Private Link service deployment would be performed by cloud team before system is handed over to a customer. This includes deploying Private Link and Standard ‘internal’ Load Balancer services in Vantage as a service virtual network. When completed, customer can then create a private endpoint in their VNet and map it to this service to the Vantage as a service resource.

Teradata recommends separate Private Link Service, Load Balancer and Private Endpoint for each Vantage as a Service offerings: SQLE, Viewpoint VM and QueryGrid

The following steps are:
Step 1 – Acquire Private Link service alias for the Vantage as a Service resource

Contact cloud team to acquire private link service alias for Vantage as a Service resource to complete private endpoint configuration.

Step 2 – Create Private Endpoint to connect to Private Link service 
 

Logon to the Azure portal and follow steps in create private endpoint. 

In the Basics tab of Create a private endpoint, enter the Subscription, Resource group, Name and Region customers virtual network to access Vantage as a Service.
 

Azure-services_8.png

Figure 7: Create a private endpoint Basics tab   

Click Next: Resource

In the Resource tab, for Connection Method choose Connect to an Azure resource by resource ID or alias and enter alias.

Azure-services_9.png

Figure 8: Create a private endpoint Resource tab   

Click Next: Configuration

In Configuration tab, enter Virtual network and Subnet

Azure-services_10.png

 

Figure 9: Create a private endpoint Configuration tab   

Click the Review + create tab or Review + create button at the bottom of the screen.

Step 3 – Receive approval from Private Link Service Vantage resource owner (cloud team)

Go to resources once deployment completes and check private endpoint approval status from cloud team.

Azure-services-11.png

 

Figure 10: Private endpoint resource overview   

Step 4 – Use Private Endpoint network interface IP to access Vantage

Once approved, click on private endpoint network interface to find IP address to connect to Vantage as a Service resource.

Azure-services-13.png

 

Figure 11: Private endpoint network interface resource overview   

Next, go to Azure Services connectivity section.

Azure Services connectivity

Some Azure Services like Power BI Service and Azure Data Factor requires a gateway component to access Vantage as a Service. Or to support data integration within your virtual network, you will need to deploy a Windows VM, install the appropriate gateway component and/or any Teradata client software.

Azure Data Factory Managed Virtual Network feature does not require a virtual network or a Self-hosted Integration Runtime gateway component. See Azure Data Factory Managed Virtual Network section for more information and skip steps 1-3 below.

This diagram displays the Azure Services and Vantage as a Service connectivity
Azure-services-14.png

Step 1 – Create Windows (VM) virtual machine

Deploy a Windows VM in customer’s virtual network VNet Peered or Private Endpoint connection to the Private Link Vantage as a Service resource to install the appropriate gateway component and/or any Teradata client software.

Logon to the Azure portal and follow steps in create a Windows virtual machine.

Ensure in Networking tab to choose virtual network peered or private link to Vantage as a Service

Next, Connect to virtual machine using RDP and perform the following: 

  • Set Enhanced Security from On to Off
  • Open Server Manager. Select Local Server in the left pane
  • Click on IE Enhanced Security Configuration setting On to bring up UI and set to Off

Azure-15.png

  • Download and install Chrome

Next, install an Azure Service gateway.


Step 2 – Install Azure Services gateway and/or Teradata client software 

Logon to your Windows VM machine and determine which gateway to install or both and/or any Teradata client software requirements.

Note gateway is required for cloud and on-premise Vantage access.

Prerequisite: 

Note Self-hosted Integration Runtime is required for using a private IP and for on-premise Vantage access.

Prerequisite: 

  • Create a Azure Data Factory as a Public Endpoint. 
  • This ensures gateway can install, access and register with the service. Next section, we will set Data Factory service network access to private and create private endpoint for service to access Vantage as a Service using private IP.
  • Logon to Azure portal from your Windows VM and open Azure Data Factory UI to install gateway.

Both gateways can co-exist on same Windows VM machine.

  • Teradata Tools and Utilities – for Teradata client software
    • Download Teradata Tools and Utilities – Windows 
    • Install components for your use case (i.e. BTEQ, Teradata Parallel Transporter Base, Teradata Access Module for Azure, .NET Data Provider for Teradata, etc…)

Logoff Azure portal and Windows VM once installation is completed.
Next, create private endpoint for an Azure Service

Step 3 – Create Private Endpoint for an Azure Service

You will need to create a Private Endpoint, Subnet and NSG rules for some Azure Services in customer’s virtual network for services to access VM running gateway, services or other products to access Vantage as a Service.

Azure Data Factory 

Set Data Factory Network access to Private endpoint

Logon to the Azure portal and open your Data Factory instance and click on Networking on the left pane.

Ensure Network access is set to Private endpoint.
 

azure-16.png

Figure 13: Azure Data Factory Network access settings   

Click Save

Create Private Endpoint for Data Factory

In the portal, go to your service instance and click Networking Private endpoint connections and click + Private endpoint. Or on the upper-left side of the portal select Create a resource and search for Private endpoint in the search box and click Create.

In the Basic tab, enter customer Subscription, existing or new Resource group, Endpoint ‘Instance’ Name and Region.

azure-17.png

Figure 14: Azure Data Factory Private endpoint Basic properties   

Click Next: Resource

In the Resource tab, choose Connection method: Connect to an Azure resource in my directory, Customer Subscription, Resource type: Microsoft.DataFactory/factories, your Data Factory Resource and Target sub-resource: dataFactory.

azure-18.png

 

Figure 15: Azure Data Factory Private endpoint Resource properties   

Click Next: Configuration

In Configuration tab, choose the Virtual network that has been peered with Vantage as a Service and where the Windows VM is running the gateway component. Should be one of the same. Choose default Subnet.

azure-19.png

Figure 16: Azure Data Factory Private endpoint Configuration properties

Click Next: Tags

Click Review+  create 

Click Create after Validation passed

Azure Data Factory Managed Virtual Network

When you create an Azure Integration Runtime (IR) within Azure Data Factory Managed Virtual Network (VNET), the integration runtime will be provisioned with the managed Virtual Network and will leverage private endpoints to securely connect to supported data stores.

Creating an Azure IR within managed Virtual Network ensures that data integration process is isolated and secure.

Currently, the managed Virtual Network is only supported in the same region as Azure Data Factory region. 
Prerequisite: Private Link connection to Vantage as a Service resource needs to exist.

This diagram displays the Azure Data Factory Managed Virtual Network and Vantage as a Service connectivity

azure-20.pngSet Data Factory Network access to Private endpoint

Logon to the Azure portal and open your Data Factory instance and click on Networking on the left pane.

Ensure Network access is set to Private endpoint.
 

azure-21.png

Figure 13: Azure Data Factory Network access settings   

Click Save

Task 1 – Acquire Private Link Service Resource ID from cloud team for your Vantage instance

You will need Resource ID when configuring your Managed Private Endpoint.

Task 2 – Create Azure Integration Runtime Managed Virtual Network

In the portal, open your Data Factory Studio UI and click on Manage icon. Next, click on Integration runtimes and click +New to create Azure Integration Runtime for a Managed Virtual Network.

azure22.png

Follow the Integration runtime setup and choose Azure, Self-HostedAzure click Continue.

Next, enter a Name and set Virtual network configuration to Enable and click Create.

azure-23.png

Once created, click on your integration runtime to ensure it is in running status.

Task 3 – Create a Managed Private endpoint in Azure Data Factory

Next, click on Managed private endpoints icon and click +New.

azure-24.png

In New managed private endpoint dialog choose Private Link Service and click Continue.

azure-25.png

In the New managed private endpoint (Private Link Service) dialog enter a Name, enter manually Resource ID provided to you by cloud team (step 1) and enter a Fully qualified domain name which will be used when creating Linked services connections to Vantage. Click Create will send private endpoint request to owner of the data source for approval.


azure-26.png

Task 4 – Receive approval from Private Link Service Vantage resource owner (cloud team)

Once the owner approves the connection, the private link is established.

azure-27.png

Task 5 – Use fully qualified domain name to access Vantage

Open Linked services and create a connection to Vantage with the domain name you entered when creating your private endpoint.

azure-28.png

Azure Blob Storage

Set Blob Storage access to Selected networks

Logon to the Azure portal and open your Blob Storage account and click on Networking on the left pane.

Ensure Network access is set to Selected networks.
azure-29.png

Figure 17: Azure Blob Storage Network access settings   

Click Save

Create Private Endpoint for Blob Storage

In the portal, go to your service instance and click Networking Private endpoint connections and click + Private endpoint. Or on the upper-left side of the portal select Create a resource and search for Private endpoint in the search box and click Create.

In the Basic tab, enter customer Subscription, existing or new Resource group, Endpoint ‘Instance’ Name and Region.

azure-30.png

Figure 18: Azure Blob Storage Private endpoint Basic properties

Click Next: Resource

In the Resource tab, choose Target sub-resource: from the dropdown list (i.e. blob, file, table, web, queue)

azure-31.png

Figure 19: Azure Blob Storage Private endpoint Resource properties

Click Next: Configuration

In Configuration tab, choose the Virtual network that has been peered with Vantage as a Service and where the Windows VM is running if required for access to your storage account. Should be one of the same. Choose default Subnet.

azure32.png

 

Figure 20: Azure Blob Storage Private endpoint Configuration properties

Click Next: Tags

Click Review+  create

Click Create after Validation passed

Azure Batch Service

You can provision the pool in a subnet of a virtual network. Before to adding a pool, the subnet must allow inbound and outbound communication from the Batch service to the compute nodes and other resources. For pools in the Virtual Machine configuration, see Network security groups: Batch default for NSG rules requirement.

  • Inbound TCP traffic on ports 29876 and 29877 from Batch service IP addresses that correspond to the BatchNodeManagement service tag.
  •  Inbound TCP traffic on port 22 (Linux nodes) or port 3389 (Windows nodes) to permit remote access. For certain types of multi-instance tasks on Linux (such as MPI), you will need to also allow SSH port 22 traffic for IPs in the subnet containing the Batch compute nodes. This may be blocked per subnet-level NSG rules.
  • Outbound traffic on any port to the virtual network. This may be amended per subnet-level NSG rules.
  • Outbound traffic on any port to the Internet. This may be amended per subnet-level NSG rules.

Create subnet to provision the pool in a virtual network.

Logon to the Azure portal and open your Virtual network and click on Subnets on the left pane.
Click + Subnet and enter a subnet Name (i.e. BatchSubnet) and choose Network security group (NSG) with required rules above.
azure100.png

Figure 21: Azure Virtual network add subnet settings  

Click Save.

Step 4 – Access Vantage

Depending on virtual network and services connectivity approach:

  • For VNet Peering connectivivty use provided Vantage as a Service IP addresses.
  • For Private Link Services connectivity use Private Endpoint network interface IP address.
  • For Services with Managed VNet support (i.e. Data Factory) use fully qualified domain name.

关于我们 Rupal Shah

Rupal Shah is a member of Teradata Partners Technical Consultant team. Prior to consulting on the Microsoft partnership, he was a technical consultant for the IBM Cognos and Oracle Hyperion partnerships. Along with his extensive experience working with business intelligence and ‘in-database’ solutions, Rupal has worked with various Teradata application organizations for whom he provided database consulting. He received his B.A. in Math and Computer Science from the University of California at San Diego, and he is currently based in San Diego. 查看所有帖子 Rupal Shah

关于我们 Shamira Joshua

Shamira Joshua is a part of the Product team at Teradata and leads cloud native integrations for Teradata’s flagship platform Vantage.

Shamira is a goal driven technologist and has a breadth of experience gained from working in companies like Cisco, Alcon labs. In her role in Teradata as a lead for cloud native integrations, Shamira is responsible for driving Vantage integrations with services & applications developed by hyperscalers to provide customers the best options to modernize as they journey to the cloud.

查看所有帖子 Shamira Joshua

随时了解情况

订阅 Teradata 的博客,获取每周向您提供的见解



我同意作为本网站提供商的Teradata天睿公司可能偶尔向我发送Teradata市场沟通电子邮件,其中包含有关产品、数据分析、活动和网络研讨会邀请的信息。我了解我可以随时通过点击我收到的任何电子邮件底部的取消订阅链接取消订阅。

您的隐私很重要。您的个人信息将根据Teradata全球隐私政策收集、存储和处理,您可以通过单击此隐私链接阅读和打印。

从 Teradata 查看更多信息