There are many concepts you must be aware, comfortable, and competent with to manage data efficiently. This section covers many data concepts that will not only help you pass the Data Engineering on Microsoft Azure exam, but also help you do the job in the real world. Keep in mind that when discussing relational structure […]
Don’t confuse data management with database management, where the focus is on the mechanics of the DBMS. When you choose to run your database on the Azure platform and select a PaaS product, then the management of that database is no longer your or your company’s responsibility. Instead, the focus here is the management of […]
You might be wondering which distribution model you should use. The answer has to do with the table category to which the table you are creating belongs; see Table 2.3. TABLE 2.3 Table category distribution matrix Category Distribution model Staging ROUND_ROBIN Fact HASH Dimension (small table) REPLICATED Dimension (large table) HASH STAGING TABLE A staging […]
This kind of data is typically media files like audio, video, or images. There is no available interface for developers to use to query the contents of media files. There are some advancements happening in the Azure Cognitive Services area, where some artificial intelligence (AI) algorithms are able the identify visual or sound patterns. Those […]
In its most common use, an index is the place you look for a key term to find a page number for the detailed explanation of that term. You will find an index at the end of this book. If you look for the term Index, you will find this page referencing it. An index […]
Introduced in Chapter 1, “Gaining the Azure Data Engineer Associate Certification,” there are two options when it comes to Azure Synapse Analytics SQL pools: serverless and dedicated. When you run in serverless, there is no need to be overly concerned about database table structures because you are querying data that already exists in other sources. […]
You will find the MMP design in Azure Synapse Analytics and Symmetric Multiprocessing (SMP) design in Azure SQL Database. MMP processors (i.e., CPUs) are allocated with dedicated compute resources like memory, while SMP shares those compute resources. Consider the fact that the retrieval of datastored in memory is less latent than when stored on disk. […]
FIGURE 2.6 Azure Cosmos DB APIs FIGURE 2.7 Azure Cosmos Data Explorer FIGURE 2.8 Azure Cosmos Data Explorer SQL query The first query returns the scenario from all the files in that container. The second query returns the first reading for a specific scenario.
Two problems that have come up regarding relational databases over the years have to do with the complexity of SQL queries and the difficulty of representing objects. The simple structured database you created in the previous exercise contains six tables. Creating a query to join three, four, or even five of those tables together would […]
