Helping you know what you don't know about data technology

Knowing What You Don't Know

I used to think I knew everything. I felt like I had the world all figured out. But then, when I went to kindergarten, I realized that I didn't know much at all. As I grew older and finished college, I regained confidence in my knowledge and again felt like I knew everything about the universe. However, as I got out into the real world of employment and failed a few times, I began to realize just how little I actually knew. That trend has continued every year of my life borne out of plenty of failures, but also from realizing just how much stuff there is to know in the world.

Socrates once said, "The only true wisdom is in knowing you know nothing," I've really taken that to heart over the years, and recently, this concept came to mind as I was learning about how PostgreSQL works. At first glance, it appeared very similar to other SQL Server products, leading me to assume that the internals and workings would be very similar. This assumption was a bit dangerous.

While there are indeed many similarities, I discovered unexpected differences, such as how concurrency is implemented. For example, SQL Server uses an entirely lockless approach to implement MVCC (Multi-Version Concurrency Control) for their in-memory objects, and I expected PostgreSQL would also. But for some parts of the isolation process, PostgreSQL implements locks. So, from what I understand currently, it seems to mean that PostgreSQL behaves pretty similar to how SQL Server does when using Read Committed Snapshot... but I am not quite sure yet. And if I was about to start a PostgreSQL project where isolation and concurrency were very important, I would not feel immediately confident to trust my knowledge or instincts.

Seeking out knowledge on topics you believe to be true is really hard, but quite often an essential, repeatable step in learning. Not only do you need to read that something is true and honestly still true, but you have to overcome that "I know everything" attitude that you are bound to have in your head. Add to that that one of the most concerning parts of "doing your research" is that when you read blogs or articles about a topic you don't fully understand, you might unknowingly accept incorrect information. Depending on how critical some knowledge is, it's essential to thoroughly test and validate the principles you learn, especially when working on complex topics.

And is that ever hard. Too often people, me included, think they know a lot more than they actually do and assume they are right without really knowing so. If you aren’t sure about a topic, take some time to learn from various sources and try to find sources you know are credible. And if you can verify that they are right, do that too.

Louis Davidson (@drsql)

Join the debate, and respond to the editorial on the forums

The article demonstrates how the use of lightweight, containerized clones removes the problem of 'mopping up' after a database migration fails part way through, particularly in database systems like MySQL that don't support DDL transaction rollback. A quick reset to the previous saved revision provides 'instant rollback'. It also demos how they make it much easier and less disruptive to do retrospective bug fixing on older migrations.

OpenAI’s search tool shows promise but lacks Google’s specialized functions and can suffer from hallucinations.

Let me be extremely clear up front, this is not my...

Temidayo Omoniyi takes us through an architecture: In today’s world, billions of data are generated daily from messaging applications like WhatsApp, financial data like the…

Introduction In this post, the third in our series on using T-SQL Snapshot Backup, I will guide you through using the new T-SQL Snapshot Backup feature in SQL Server... The...

Learn how to create a PowerShell script that can listen to your voice and respond with spoken words.

Have you ever surfed the internet and felt overwhelmed by personalized advertisements that appear to know your every desire? Have you ever been amazed at how accurate some weather forecasts can be? These actions are based on data processing.

An entire post on choosing the correct data types ...

Let’s say you built a web app and it became glob...

Effective DevOps infrastructure management is key to accelerating development cycles, enhancing collaboration, and maintaining security while aligning with organizational goals and customer needs.

Sandeep Pawar talks about a Spark connector: I am not sharing anything new. The spark data warehouse connector has been available for a couple months now. It…

In this post I show two different ways to authenticate when calling APIs. One way to authenticate with a service principal and another with a Microsoft Entra ID user account that has multifactor authentication (MFA) disabled. As per the below diagram.

We talked about Git integration with Azure DevOps in the past, but now you can integrate GitHub with Microsoft Fabric! Once things are configured most things are the same because it’s just Git! Adam walks you through it!

Indexes in MySQL are one of the primary ways to enhance query performance and they are especially useful when the primary use case of our project refers to reading data stored in a database. We‘ve already told you about the nuanced world of indexes in MySQL – and there we‘ve mentioned that MySQL has multiple types of SQL indexes available for you to choose from.

These are my least favorite mysteries to resolve. ...

When working with data, I usually have an idea of ...

In a previous tip on SQL Server 2016 Query Store Introduction, we've seen how to enable Query Store and find out the top resource consumers. We've experienced high CPU utilization recently and wanted to know the root cause and how to fix it. How can we use Query Store to achieve this?

Annoyances With Filtered Indexes And Computed Colu...

This SQL tutorial focuses on the differences and similarities of these operations. Before beginning, some important concepts related to this topic will be explained. In particular, background information about the fill factor and statistics will be provided, as these concepts are referred to while discussing Index Rebuild and Reorganization operations.

When To Use DBCC DROPCLEANBUFFERS When You’re Tuning SQL Server Queries Thanks for watching! Going Further If this is the kind of SQL Server stuff you love learning about,...

A Foreign Key Join Elimination Limitation In SQL S...

Range queries are a very common task in SQL: selecting dates, numbers, or even text values that fall within some specified range. For instance: all loan applications for the month of March, or all sales transactions between $50 and $500. SQL developers are generally adept at writing such queries and indexing them to ensure they perform well.

In this episode, we're going to look into how we can read that Always Encrypted data (On Prem, with the Key secured by Azure Key Vault), using Power BI and PowerBI Desktop.

The DAX Query View for Power BI got some nice updates and a special item for those on a Mac! Zoe Douglas walks us through what we can do!

As a data analyst, my switch from Excel to SQL was revolutionary. This article reveals how an online SQL basics course dramatically improved my workflow. I've always depended on... The...

The fifth episode of Simple Talks Podcast is out! This episode was recorded in Austin, where we celebrated the Redgate 25th Birthday as a company. Grant, Ryan, and I grabbed a conference room. Ryan and I also grabbed some cowboy hats that were in the office.

Hello SSMS users! A short post today to remind folks to check out the feedback site regularly. As I mentioned in a previous post, I started a Friday Feedback...