DataWarehouseAutomation 1.2.3

.NET Standard 2.0 This package targets .NET Standard 2.0. The package is compatible with this framework or higher.
There is a newer version of this package available.
See the version list below for details. 
dotnet add package DataWarehouseAutomation --version 1.2.3                

                    
                

            
NuGet\Install-Package DataWarehouseAutomation -Version 1.2.3
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package. 
<PackageReference Include="DataWarehouseAutomation" Version="1.2.3" />
For projects that support PackageReference, copy this XML node into the project file to reference the package. 
paket add DataWarehouseAutomation --version 1.2.3                

                    
                

            
#r "nuget: DataWarehouseAutomation, 1.2.3"
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package. 
// Install DataWarehouseAutomation as a Cake Addin
#addin nuget:?package=DataWarehouseAutomation&version=1.2.3

// Install DataWarehouseAutomation as a Cake Tool
#tool nuget:?package=DataWarehouseAutomation&version=1.2.3

Generation Metadata Schema for Data Warehouse Automation

Intent

To provide a collaborative space to discuss an exchange format concerning ETL generation metadata, supporting Data Warehouse Automation. This adapter should contain all metadata necessary to generate the transformation logic for a Data Warehouse solution.

The following directories have been set up:

  • Generic interface, containing the Json schema definition.
  • Class Library (DataWarehouseAutomation) containing the object model for deserialisation, as well as various utility classes such as validation of files against the Json schema definition.
  • Code examples (examples_handlebars), containing C# examples using the generic interface for various purposes.
  • Regression test project (test_project)
Hypothesis

Across most, if not all, metadata models there is a core set of information that is required for any generation of ETL. If we can separate this from the UI / management of metadata we could have an exchange format that allows anyone to 'plug in' their own desired technology.

As an example 'TEAM' has the intent to separate UI with a view of limiting data entry and validation, but does not focus on SQL generation. Rather, these functions are separated by an adapter that is accessible as Json or database view.

Requirements

The fundamental requirements of the metadata adapter are:

  • Containing all metadata required to generate ETL output. This notably includes:
    • source-to-target mappings
    • physical model metadata (columns and tables, data types etc.)
    • connectivity information, or proxy
  • Text-based to support version control
Background

In the Data Warehouse Automation (DWA) domain there are many specialists (i.e. ETL developers, Data Warehouse and Data Architects, BI analysts etc.) who have been, or are working on, proprietary meta models to support forward-engineering of code and designs.

Some of these are built inside existing tools (i.e. ERwin, Powerdesigner) using SDKs or macros. Others use different development frameworks (.net, Java) and most use differently modelled repositories or file formats to persist data on disk.

This is in addition to the many off-the-shelf DWA platforms, each of which has their own repository and format as well.

In the broader sense of meritocracy, it is worth pursuing if a common exchange format for metadata can be defined in a way that any developer can develop to in whatever technology or way their passion drives them.

Working guidelines

For any change, create a new branch (no direct commits to master branch).

Product Compatible and additional computed target framework versions.
.NET net5.0 was computed.  net5.0-windows was computed.  net6.0 was computed.  net6.0-android was computed.  net6.0-ios was computed.  net6.0-maccatalyst was computed.  net6.0-macos was computed.  net6.0-tvos was computed.  net6.0-windows was computed.  net7.0 was computed.  net7.0-android was computed.  net7.0-ios was computed.  net7.0-maccatalyst was computed.  net7.0-macos was computed.  net7.0-tvos was computed.  net7.0-windows was computed.  net8.0 was computed.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed. 
.NET Core netcoreapp2.0 was computed.  netcoreapp2.1 was computed.  netcoreapp2.2 was computed.  netcoreapp3.0 was computed.  netcoreapp3.1 was computed. 
.NET Standard netstandard2.0 is compatible.  netstandard2.1 was computed. 
.NET Framework net461 was computed.  net462 was computed.  net463 was computed.  net47 was computed.  net471 was computed.  net472 was computed.  net48 was computed.  net481 was computed. 
MonoAndroid monoandroid was computed. 
MonoMac monomac was computed. 
MonoTouch monotouch was computed. 
Tizen tizen40 was computed.  tizen60 was computed. 
Xamarin.iOS xamarinios was computed. 
Xamarin.Mac xamarinmac was computed. 
Xamarin.TVOS xamarintvos was computed. 
Xamarin.WatchOS xamarinwatchos was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
2.1.0 112 7/8/2024
2.0.3 136 4/21/2024
2.0.2 279 11/12/2023
2.0.1 143 11/7/2023
2.0.0 155 9/17/2023
1.3.4 241 8/18/2023
1.3.3 168 8/16/2023
1.3.2 168 8/16/2023
1.3.1 172 5/15/2023
1.2.5 315 3/10/2023
1.2.4 260 2/22/2023
1.2.3 385 10/31/2022
1.2.2 382 10/26/2022
1.2.1 365 10/26/2022
1.2.0 383 10/13/2022