Data Quality CSV

I have been working on automated data ingestion activity on snowflake. The python code uses Snowflake INFER_SCHEMA with Evolution Changes to load the data. data-load with schema evolution The system is set up to automatically figure out the file metadata schema from a group of staged data files that contain semi-structured data, and then get the column definitions. The GENERATE_COLUMN_DESCRIPTION function uses the result of the INFER_SCHEMA function to make it easier to create new tables, external tables, or views (using the right CREATE command) based on the column definitions of the staged files....

April 26, 2024 · 2 min

Basic Notes on Containerization

Few Tips : Whenever config-map is updated, you need to manually restart the services to update the changes Ephemeral Volumes exist for the lifespan of a Pod. Persistent Volumes preserve data beyond the lifespan of a Pod. Volume Types emptyDir - Empty directory in Pod with read/write access. Only persisted for the lifespan of a Pod. A good choice for cache implementations or data exchange between containers of a Pod. hostPath: File or directory from the host node’s filesystem....

March 13, 2024 · 3 min

K8s - CKAD Exam Tips and Notes

The CKAD exam is about more than just your knowledge of Kubernetes. It also tests your ability to complete tasks quickly. Speed ​​and efficiency are key to passing this test. This exam truly tests the pratical working knowledge and your proficiency. Please read the important instructions about device you use for taking exam, protocor requirements, exam details etc Few Tips : Please run PSI Online Proctoring System Check before the exam to ensure no surprises on exam day....

February 14, 2024 · 7 min

Geospatial Tech Radar - 2024

Recently came across the geospatial tech radar article published by Element84. This was widely recieved with in gis community and many conversations/discussions happened in social media (twitter/linked-in). The radar aims to highlight technologies that are proven and mature (Adopt), technologies that are ready for use but not fully proven (Trial), technologies worth looking at closely but not necessarily for trial (Assess),and technologies that haven’t happened yet but have forthcoming developments (Watch)....

February 13, 2024 · 2 min