Airflow Oracle connection and issues.

 Issue: DPY-4000: unable to find "" in /opt/oracle/instantclient/network/admin/tnsnames.ora

Explanation: Helm Built-in Objects.

In this section we are going to learn about helm built-in objects.

So these objects are passed into a helm template from the helm template engine.

And these objects can be simple and have just one value or they can contain other objects or functions.

So these helm objects start with the root object, and under this root object, we are going to have

these six built in objects.

So root object, you can call it as dot, or you can call it as period, or even the root object.

And this is the top level helm object at the top level.

Helm initializes this dot to an object with keys like dot release, dot value, dot chart, dot capabilities,

dot template and then dot file.

Right.

And we are going to have one one small small demos for each of these objects to including the root object

for the root object.

We are going to go to notes, dot txt and empty.

Whatever is there.

We are going to create a simple helm chart and then we are going to remove everything from the notes

dot txt and put this simple code root object colon.

And with these flower brackets we are going to put Dot here and then we are going to execute this using

helm install and then dry run command and see what is the output of this respective root object and

what you will get.

You will get an entire object map.

Okay?

So in this map you will find the capabilities object, you will find the chart object, you will find

the files object, you will find the release object and subcharts object if any present if not template,

object and values object.

Okay, so whatever you are seeing here, all these things you are seeing here, right?

So that's the thing, which means whenever you use dot your entire helm.

Complete helm chart is loaded here.

Or you can say the helm object.

The root object is loaded here, so it will have all the information about your helm chart inside that

root object.

Now, before going and then implementing this root object demo, we need to understand one more thing.

So if you remember, we have discussed about Notes.txt in the folder structure section, which is our

previous demo in that we have discussed three important points.

One is Notes.txt file is rendered same as regular template deployment, deployment.yaml or service dot

yaml by helm, which means whenever you run the helm, install command.

So whatever the logic you define in notes.txt or whatever the helm logic you define in Deployment.yaml,

so both are treated same and it is also rendered as same.

The main difference is after notes.jsx txt rendered, it's not, it is not sent to the Kubernetes cluster.

So example, once you run the helm, install command.

So all the replacements related to helm logic will be happened in the deployment.yaml and that is sent

to the Kubernetes cluster for deployment or service or whatever it is there, right?

So they will be sent and then they will be deployed in the Kubernetes cluster.

But in the notes.txt, whatever is there, it is going to be displayed as command line output for us.

Okay.

Whenever.

Which means once the helm install command is completed in the command line output with nodes data,

you are going to get it.

So which means you can define any type of logic in notes.txt and immediately you can test with helm

install with dry run command.

So in fact you can use Notes.txt during the development of your helm chart as a test bed, which means

you can go there and then write the logic and run the helm, install dry run and if it is working or

not, immediately in the command line window, you can test it.

And if that code snippet is working then only you can take it and then replace in your deployment.yaml

or whatever the template you want to in which template you want to put it.

That one.

So now we are going to do the same way in the notes.txt.

We are going to put the root object and run with helm, install dry run and then see what happens.

So let's go back.

And we'll go to our GitHub repository here and we are in section 11.

Helm Dev built in objects.

So this is our first development demo.

So from demo 11 onwards, we are starting the base development for our or starting to learn the helm

development here.

So we'll go here and we'll start with step zero two, which is create a simple chart and clean up the

Notes.txt.

So let's go back here in the terminal and in the helm master class section.

So I'll say clear.

I'll go to the section 11 helm dev built in objects and here I'll run the command helm, create and

provide the chart name.

So it is my chart name.

Anything I can put, I can say my chart one or my first chart or whatever the chart name.

So I'm going to give more relevant name.

So I'll say this is a built in objects demo.

I'll say built in objects, right?

So that is my helm chart name.

And now I'll go inside this built in objects and I'll put it this way and come back to the Visual Studio

code.

And in Section 11 helm dev built in objects in built in objects folder.

Right.

This is my chart.

And inside that I'll go to templates and inside that I'll go to notes, dot, txt and whatever.

So whatever is there inside this notes dot txt I'm going to clean up and I will save it.

And now here I can start writing the base template code.

Right?

So the first thing is in your helm templates, right in your helm language or whatever we call it.

Right.

The first thing is how to create a comment.

To create a comment.

So we need to take two flower brackets and say slash star and star slash.

And in between this, you can add any comment here.

So you can add it as like something like root or dot or period object.

So you can say root or dot or period object, right?

And now let's run the helm.

We are in the Built in Objects folder, which is your chart directory.

There you are running the command helm install and provide the release name so you can say my app 101

or something.

And where is your chart directory?

You are in the current chart directory itself.

You are using dot.

So this dot is not your this dot.

Okay.

What we are talking about.

Right?

So this is just your current directory dot you are in built in objects in your chart directory.

So you're saying this is just a current directory.

Okay.

And then we need to provide dry run.

Right?

And let's see what happens.

So it has printed the templates for us and there is no notes printed for us here at the end.

Right.

So that's what we need to verify.

So notes dot txt is executed at the end here and then it displays the notes here and it is not displayed

because it is empty.

And whatever you have defined here it is considered as comment.

So it is not printed here.

So now let's go to the next thing.

So we'll say root object or whatever the name we want to put.

So root object and say and just leave it like this and one more time, run the helm.

Install my app 101. is nothing but your current directory you are in built in objects and dry run and

you can see you got the notes and root object.

And here we still didn't provide any data here.

Okay.

So it didn't print the comment, but it printed the text.

Whatever you have written now we will again put double quotes, the flower brackets and put dot here.

Right.

And now let's see what happens.

Right?

So I'll run the command helm, install my app 101 and dot is the current directory.

I am inside the chart directory now you can see there is a lot of data came in.

So this is the data for our root object.

Okay.

So let me copy this so that we get an idea what it has printed here.

Right.

So now I'll take something called.

Okay.

I can create a file named root object content.

Okay.

So I'll say root object content outside my chart directory, inside my Levon Helm chart structure.

I have pasted it here.

Right.

It is a single line if you see here, right?

It's completely a single line with map.

Right?

So this is the map and this is the capabilities object and this is the charts object.

And in charts object.

So this is the chart name you can say, right.

So this is the chart version.

You can say this is the chart description, right?

And this is the chart version.

So like this, all the chart related information is present in chart related object.

So let me go go here and you will find here the files object.

So files object will load all the files here.

So before going in we will discuss this chart objects on a high level.

So let's go to built in objects and go to chart dot yaml in chart dot yaml.

You can see api version is v2 and you can see here it is v2 and you can see the description of the chart,

a helm chart for Kubernetes.

So you can see that a helm chart for Kubernetes and you can also see the chart version 010 and you can

see that here, version 010.

And what is the chart name built in objects?

So you can see it here.

So all the chart object related information is present in chart and that you can load using dot chart

actually.

But if you want to use route object, it is under the dot.

All these objects are present.

Now this is a file, this is a map.

And once this file object is going to be binary data and we are going to have lot of files here, right?

So we'll go here, here, here, and we'll play with the sub chart.

Okay.

This is sub chart before sub chart will go here and once those numbers are completed we'll see any other

thing.

See, this is a release, right?

And this is sub chart and this is template.

And after in template and after template, we have something here which is values.

Okay?

We can see values and after values.

Do we see anything?

Nothing.

We'll close the main map here.

Okay, so this is the route object.

Where is it?

Map.

Okay.

And it has all the objects.

So in the root object.

So if you see.

This all these things capabilities chart files release Subchart template values.

All the information of your entire helm chart is present inside this root object.

So that is what we have printed here.

So now this is completed.

The demo about root object is completed.

But another important thing we need to remember here is we use if we use Dot inside range and with actions

range and with our flow control actions from helm perspective and these things, when we are discussing

one more time, we are going to discuss about this dot.

But here also I want to reiterate because it's a very important point.

So if we use Dot inside a range and with actions, it is going to be current context in that respective

block and not the root helm object.

So which means in the helm template, whatever you are seeing here, you take just a deployment.yaml

right?

So in this deployment.yaml, wherever you use Dot, which means you are passing your complete helm template

there, right?

So or you are passing the complete helm chart there you can call it as a root object or a period or

a dot.

So you can see it here include so and so something, something.

And you are passing your here, you can see you are passing the dot, which is your root object here.

Also you can see you are passing the dot root object.

So like this, if you see here in the width, right in width, if you see here, if you are passing

dot means it is the current context.

Current context means dot values, dot image, pull secrets is the current context here.

So if you pass anything inside width or anything inside range that is current context inside the that

respective block and nothing and it is not the root object any anywhere else.

If you pass then it is going to be 100%.

It is the root object.

So that's the exception, which means when you pass in either width or inside the range related blocks,

you need to ensure that that is not the root object, that is the current context object, except that

everywhere, wherever you pass that it is going to be root object only inside your templates, wherever

it is.

So this completes the root object demo.

In the next lecture we will learn about release, object and other objects.

So I'll see you in the next lecture.

Until then, bye bye.

Thank you.

information alert

Schedule learning time

Learning a little each day adds up. Research shows that students who make learning a habit are more likely to reach their goals. Set time aside to learn and get reminders using your learning scheduler.

About this course

Create a simple Helm chart with Example.

Helm Charts:

The Helm package manager for Kubernetes.

Tiller is the server portion of helm and you need to install it in you Kubernetes cluster using the below command.

Command: helm init

First you need to Download helm from the below link based on operating system:

https://github.com/helm/helm/releases

 

Set path in windows to run the helm commands through CLI:

Copy full path where you extracted downloaded file from above link:

Goto environmental variable and select path and click edit :

Click new and add the helm path:

Now you can restart the system and goto command prompt check with helm command:

 

Now tiller is in your cluster then net you need to role bind the cluster admin permissions to deploy the services.

 

Create the helm chart:

helm create <CHART-NAME>

helm create basechart

Observation:

1. It will create a Helm Chart template

2. We can call it like a helm chart created from a default starter chart

Inside the directory there is deployment, service, ingress manifests in the templates folder.

Chart.yaml is the file which contains the information about the metadata(application,library etc..) and basic API information about the Chart. Following is a sample file of helm Chart.yaml

Example:

 

Values.yaml:

Values.yaml file will have the values like Image parameters, Dependencies, Configurations. This will always override the values which are defined in the chart file.  Here are some samples.

We can also pass the values to the commands from external files

 Examples: helm install -f path/to/myvals.yaml  ./mychart

                    helm upgrade --install my-airflow apache-airflow/airflow  --values .\values.yaml

.helmignore: contains pattern to ignore when packaging helm charts.

 

Templates Folder.

We have a template/ folder created under the same directory when you execute the helm create command. This particular folder will have the default folders as mentioned below.

• NOTES.txt
• deployment.yaml
• service.yaml
• _helper.tpl

NOTE.txt file is like the document file which will be displayed when you run the helm install command.

Deployment.yaml will have the manifest to create Kubernetes Deployment objects.

Service.yaml is the file which will have the manifest for creating a service endpoint you’re your deployments.

_helpers.tpl file will have the re-usable component throughout the chart.

Charts/ folder:

So, charts/ folder will carry any of the charts that the user is going to create. We can say it as dependencies of the main chart.yaml file. Say for example, If we are going to create mysql DB using Kubernetes and we are going to deploy the same using the helm charts. So, we can create a directory structure as mentioned below.

Example structure:

Charts.yaml

..

..

Charts/

    mysql/

        Chart.yaml

 

The Chart.yaml file defined inside the subdirectories is called as sub chart. And every file can fetch its own values.yaml.

 

How to create a Template:

As the first step of creating the Helm chart, we will create a ConfigMap Template, we need to create a template file called configmaps.yaml file. This file should contain the following lines.

apiVersion: v1

kind: ConfigMap

metadata:

  name: mychart-configmap

data:

  myvalue: "Sample Config Map"

 

 

Basically, this file will simply store the information that will help Kubernetes to have its object to store configuration data. In this file, myvalue: “sample Config Map” is the key value pair that will be sent to the Kubernetes.

To apply the above configuration to the cluster, we need to run the following command.

$ helm install helm-demo-configmap ./my-chart

So, when you check the Kubernetes cluster, you can see the ConfigMap created by using the Helm chart template that we created now.

 

Tests folder:

We write tests we validate that validate our chart works as expected when it is installed.

These tests also help the chart consumer understand what this helm chart supposed to do.

Verify if application is up and running after deploying the chart:

     Command: helm test release-name

 

Helm Variables:

Variables are assigned with a special assignment operator :=

How we can define a variable?

How we can reference a variable?

Below commands are used to perform the actions  uninstall, status, history, rollback:

Helm status release-name:

Helm uninstall release-name 

helm install . --dry-run –debug

helm upgrade what-the-helm .

helm history what-the-helm

helm rollback what-the-helm 1

helm chart folder structure:

Helm useful commands:

• helm completion - generate autocompletion scripts for the specified shell
• helm create - create a new chart with the given name
• helm dependency - manage a chart's dependencies
• helm env - helm client environment information
• helm get - download extended information of a named release
• helm history - fetch release history
• helm install - install a chart
• helm lint - examine a chart for possible issues
• helm list - list releases
• helm package - package a chart directory into a chart archive
• helm plugin - install, list, or uninstall Helm plugins
• helm pull - download a chart from a repository and (optionally) unpack it in local directory
• helm push - push a chart to remote
• helm registry - login to or logout from a registry
• helm repo - add, list, remove, update, and index chart repositories
• helm rollback - roll back a release to a previous revision
• helm search - search for a keyword in charts
• helm show - show information of a chart
• helm status - display the status of the named release
• helm template - locally render templates
• helm test - run tests for a release
• helm uninstall - uninstall a release
• helm upgrade - upgrade a release
• helm verify - verify that a chart at the given path has been signed and is valid
• helm version - print the client version information

 

Airflow installation through helm:

# add helm repo

helm repo add apache-airflow https://airflow.apache.org

# get project

PROJECT=$(oc project -q)

 

# get openshift uid/gid range

CHART_UID=$(oc get project sbipoc-dev -o jsonpath="{['metadata.annotations.openshift\.io/sa\.scc\.uid-range']}" | sed "s@/.*@@")

CHART_GID=$(oc get project sbipoc-dev -o jsonpath="{['metadata.annotations.openshift\.io/sa\.scc\.supplemental-groups']}" | sed "s@/.*@@")

 

echo "UID/GID: ${CHART_UID}/${CHART_GID}"

 

# copy and edit values.yaml

cp example_values.yaml values.yaml

 

# edit values.yaml

 

# install via helm

Install the apache airflow with custom values.yaml file.

helm upgrade \

    --install airflow apache-airflow/airflow \

    --namespace ${PROJECT} \

    --version 1.10.0 \

    --set uid=${CHART_UID} \

    --set gid=${CHART_GID} \

    --set redis.securityContext.runAsUser=${CHART_UID} \

    --set postgresql.volumePermissions.enabled=true

     --values ./values.yaml

 

 

 

Create Routes:

# create route for airflow

We need a one route to host the application outside world. Please use below command to create a route for service.

oc create route edge \

    --service=airflow-webserver \

    --insecure-policy=Redirect \

    --port=8080

 

# create route for airflow flower

oc create route edge \

    --service=airflow-flower \

    --insecure-policy=Redirect \

    --port=5555

 

# confirm routes

oc get routes-à this command helps to get the route url to access the webpage.

 

 

https://artifacthub.io/packages/helm/airflow-helm/airflow/8.5.1