Integrating Megaport with Snowflake on AWS

You can use Megaport to create an AWS Direct Connect Layer 2 connection between your on-premises or colocation-based infrastructure and your Snowflake environment on AWS.

Before you begin, ensure that you have created a Megaport. After you create the Megaport, you can connect a Virtual Cross Connect (VXC) from the Megaport to the virtual gateway associated with the AWS VPC infrastructure. A VXC is a point-to-point Ethernet connection between an A-End (your Megaport) and a B-End (in this case, your AWS instance).

If you aren’t a Megaport customer, you can create a 1 Gbps or 10 Gbps Megaport in one of our global data centers/points of presence. If your company isn’t located in one of our PoPs, you can procure a last mile circuit to one of the sites to connect to Megaport. Contact Megaport for more information.

This figure shows a high level topology diagram of a solution integrating Megaport with Snowflake Data Warehousing on AWS using a single connection.

Setting up your Snowflake environment

To set up your Snowflake environment, you will:

• Log in to Snowflake
• Create Snowflake objects
• Stage the data files
• Copy data into the target table
• Query the loaded data

For details on setting up Snowflake in an AWS environment, see Snowflake Prerequisites.

To ensure that data transfer between your Snowflake solution and your VPC remains secure, you can use AWS PrivateLink to connect the two environments. PrivateLink is an AWS service that lets you implement direct and secure connectivity between AWS VPCs. This allows your data to stay within the AWS infrastructure without being exposed to the internet which reduces some of the security risks related to having data traversing the internet.

Integrating Megaport with Snowflake Data Warehousing on AWS

You can use the Megaport Portal to create the VXC to the Snowflake on AWS environment.

1. In the Megaport Portal, go to the Services page and select the Megaport you want to use. If you haven’t already created a Megaport, see Creating a Megaport.

2. Add an AWS connection for the Megaport.
   If this is the first connection for the Megaport, click the AWS tile. The tile is a shortcut to the configuration page. Alternatively, click +Connection, click Cloud, and click AWS.

3. For AWS Connection Type, click Hosted VIF or Hosted Connection and click Next.
   For this example, we will click Hosted Connection.
   

   Next, you’ll create a new VXC.

   

4. In the Select Destination Port list, select the AWS region and the interconnection point for your connection and click Next.

   You can use the Country filter to narrow the selection.

5. Specify these connection details:

   • Name your connection – The name of your VXC to be shown in the Megaport Portal.
   • Invoice Reference – This is an optional field. It can be any text, such as a PO number or billing reference number.
   • Rate Limit – The speed of your connection in Mbps. You must choose from the provided bandwidth options (50 Mbps to 10 Gbps). The sum of all hosted virtual VXCs to a service can exceed the port capacity (1 or 10 Gpbs), however the total aggregate will never burst beyond the port capacity.
   • Preferred A-End VLAN – Optionally, specify an unused VLAN ID for this connection. This VLAN ID must be a unique ID on this Megaport and can range from 2 to 4093. If you specify a VLAN ID that is already in use, Megaport will assign the next closest available value. If you don’t specify a value, Megaport will assign one.

   

6. Click Next.

7. Specify the cloud details:

   • AWS Connection Name – This is a text field and will be the name of your virtual interface that appears in the AWS console. For easy mapping, use the same name for this field as you did for the VXC name on the previous screen.
   • AWS Account ID – This is the ID of the account you want to connect. You can find this value in the Account Settings section of your AWS console.
     

8. Click Next.

9. Review the connection details and click Add VXC.
   
10. Click Order.
    
11. Click Order Now.
    Your work in the Megaport Portal is complete. Next, you will connect the new VXC to your AWS environment.
12. In approximately 2 minutes, log in to your AWS account. The VXC you implemented will appear in your Direct Connect under Connections.
13. Click Create connection.
    You will then need to connect to a Direct Connect Gateway. For details on connecting to various types of gateways, see Working with Direct Connect gateways - AWS Direct Connect.

Redundancy

To achieve redundancy for the connectivity portion of this solution, establish an additional VXC to a diverse edge router within the AWS environment. Follow the Integrating Megaport with Snowflake Data Warehousing on AWS procedure from step 3, and choose the alternate Diversity Zone (identified by the orange circle icon) when choosing the AWS region.

Complete the procedure, including the required steps within the AWS console.

This figure shows a high-level topology diagram of a solution integrating Megaport with diverse connections into the Snowflake Data Warehousing on AWS infrastructure.

For additional physical redundancy for the connectivity, you can implement the VXCs on separate Megaports:

Background information

Snowflake is a Software-as-a-Service (SaaS) data warehouse service that is purpose-built for the cloud. The infrastructure is built within the AWS cloud and allows for fast and simple implementation and integration with your AWS VPC environment and on-premise infrastructure. In a Snowflake on AWS environment, you can take advantage of benefits such as elasticity and scalability while only paying for the resources that you use. It provides metadata management, security of data, and an ANSI compliant engine on a resilient and redundant platform that allows you to store, query, and analyze all of your data in one place. Snowflake is available within the AWS Marketplace; it competes with services that are offered directly by AWS, but can offer more functionality at lower costs. We recommend that you review available options to determine which is best for your business requirements.

For more information on Snowflake, see these additional links:

• The Modern Cloud Data Platform Built for Any Cloud
• AWS PrivateLink for Snowflake: No Internet Required
• AWS Marketplace: Snowflake On Demand - Premier
• Data Warehouse Architecture