LearnwithVishnu
Basics to Production to Architect
← Home
📊MIS & Data Analytics
BeginnerAnalystAdvanced
Excel · SQL · Python (Pandas) · Power BI · Airflow · Reporting Automation
OverviewExcelSQLPythonPower BIAirflowRoadmapInterview Q&As

📊 What is MIS and Why It Matters

MIS = Management Information System. It is the process of collecting, processing, and presenting data so managers can make better decisions. In practice it means: someone asks "what were our sales last month by region?" and you answer with a table, chart or dashboard — not a guess.

Without MISWith MIS
Managers guess based on intuitionDecisions backed by accurate data
Reports take hours to prepare manuallyAutomated reports delivered every morning
Data sits in silos across departmentsOne source of truth everyone accesses
Errors in manual Excel copy-pastePython scripts eliminate human error
Old data by the time report is readyLive dashboards updated in real time

The MIS toolkit — what each tool does and when you use it

ToolWhat it doesWhen to useReplaces
ExcelCalculations, charts, pivot tables, quick reportsOne-off analysis, small datasets (<100K rows), sharing with non-technical staffPaper reports, calculators
SQLQuery databases directly — filter, join, aggregate millions of rowsWhen data lives in a database and you need specific slices fastWaiting for IT to export data
Python (Pandas)Automate repetitive Excel/data tasks, process large files, send reportsSame task runs daily/weekly, large files slow Excel down, need automationManual Excel work, VBA macros
Power BILive interactive dashboards shared across the organisationManagement needs self-service reporting, data changes frequentlyStatic PowerPoint charts, emailed Excel files
AirflowSchedule and orchestrate data pipelines — run SQL + Python automaticallyPipelines that must run on a schedule without human interventionWindows Task Scheduler, manual running of scripts

📊 Excel — Pivot Tables, VLOOKUP, Advanced Formulas, Macros

Pivot Tables — the most important Excel skill

A pivot table summarises thousands of rows into a meaningful table in seconds. Insert → PivotTable. Drag fields: Rows (what you want to group by), Values (what you want to sum/count), Filters (what you want to narrow down), Columns (optional second grouping).

TaskRowsValuesFilter
Sales by regionRegionSum of SalesYear=2025
Count of patients by diagnosisDiagnosisCount of PatientIDMonth=Jan
Average salary by departmentDepartmentAverage of Salary

VLOOKUP vs HLOOKUP vs INDEX/MATCH

FunctionSyntaxWhenLimitation
VLOOKUP=VLOOKUP(lookup_value, table_array, col_index, FALSE)Look up a value in the leftmost column of a tableCan only look left-to-right; breaks if columns are inserted
HLOOKUP=HLOOKUP(lookup_value, table_array, row_index, FALSE)Look up across the top row of a table (horizontal)Less common; rarely used today
INDEX/MATCH=INDEX(return_range, MATCH(lookup_value, lookup_range, 0))Any direction lookup, more flexible than VLOOKUPSlightly more complex syntax but worth learning
XLOOKUP=XLOOKUP(lookup_value, lookup_array, return_array)Modern replacement for VLOOKUP — Excel 365 onlyNot available in older Excel versions

Essential formulas every MIS analyst must know

FormulaWhat it doesExample
SUMIFSum with one condition=SUMIF(A:A,"North",B:B) — sum column B where column A = "North"
SUMIFSSum with multiple conditions=SUMIFS(Sales,Region,"North",Month,"Jan")
COUNTIF/COUNTIFSCount with conditions=COUNTIF(Status,"Completed")
IFERRORCatch formula errors gracefully=IFERROR(VLOOKUP(A2,Sheet2!A:B,2,0),"Not Found")
TEXTFormat numbers/dates as text=TEXT(A2,"DD-MMM-YYYY") → "15-Jan-2025"
LEFT/RIGHT/MIDExtract text=LEFT(A2,3) → first 3 characters
TRIM/CLEANRemove extra spaces/characters=TRIM(A2) removes leading/trailing spaces
UNIQUE/FILTERDynamic array functions (Excel 365)=UNIQUE(A:A) — deduplicated list automatically

Macros and VBA — when to use them

Macros record your clicks and replay them. Use for: formatting reports the same way every time, copying data between sheets on a schedule, sending email alerts. Record with: Developer tab → Record Macro. Edit in VBA: Alt+F11. Simple VBA to loop through rows and colour cells based on value — saves hours of manual formatting.

Data Validation and Named Ranges

Data Validation: Data → Data Validation → List. Prevents wrong data entry — dropdown of valid choices. Named Ranges: Select a range → Name Box (top left) → type a name. Use =SalesData instead of =Sheet1!A2:A500. Makes formulas readable and maintainable.

🗄️ SQL — Querying Databases from Basic to Advanced

The 5 clauses you need for 90% of queries

ClausePurposeExample
SELECTChoose which columns to returnSELECT name, salary, department
FROMWhich table to queryFROM employees
WHEREFilter rows by conditionWHERE department = 'Sales' AND salary > 50000
GROUP BYAggregate rows by a categoryGROUP BY department
ORDER BYSort the resultsORDER BY salary DESC

JOINs — combining data from multiple tables

JOIN typeReturnsUse when
INNER JOINOnly matching rows in BOTH tablesYou want records that exist in both (most common)
LEFT JOINAll rows from left table + matching from rightKeep all left records even if no match (show NULLs)
RIGHT JOINAll rows from right table + matching from leftRare — can usually rewrite as LEFT JOIN
FULL OUTER JOINAll rows from both tablesFind records that exist in one table but not the other

Aggregate functions — the backbone of MIS reporting

FunctionWhat it doesExample query
COUNT(*)Count all rowsSELECT department, COUNT(*) as headcount FROM employees GROUP BY department
SUM(col)Total a numeric columnSELECT region, SUM(sales) as total_sales FROM orders GROUP BY region
AVG(col)Average valueSELECT AVG(salary) FROM employees WHERE department='Finance'
MAX/MIN(col)Highest/lowest valueSELECT MAX(order_date) as latest_order FROM orders

Date functions — essential for MIS time-based reporting

FunctionMySQL/SQL ServerResult
Today's dateGETDATE() / NOW()2025-06-05
Extract monthMONTH(date_col)6 (June)
Format dateFORMAT(date,'YYYY-MM')2025-06
Date differenceDATEDIFF(day, start, end)Number of days
Last N daysWHERE date >= DATEADD(day,-30,GETDATE())Filter last 30 days

Subqueries and CTEs — cleaner complex queries

-- CTE example: find departments above average salary
WITH dept_avg AS (
  SELECT department, AVG(salary) as avg_sal
  FROM employees
  GROUP BY department
)
SELECT * FROM dept_avg
WHERE avg_sal > (SELECT AVG(salary) FROM employees)
ORDER BY avg_sal DESC;

🐍 Python for MIS — Automate Excel, SQL, and Reports

6-step automation framework for any MIS task

StepCodeWhat it does
1. Importimport pandas as pdLoad the Pandas library
2. Read datadf = pd.read_excel("sales.xlsx")Load Excel/CSV into a DataFrame
3. Cleandf.dropna() / df.fillna(0)Remove or fill blank values
4. Transformdf.groupby("Region")["Sales"].sum()Summarise like a pivot table
5. Analysedf[df["Sales"] > 10000]Filter rows by condition
6. Outputdf.to_excel("output.xlsx", index=False)Save result

Complete automation script — monthly sales report

import pandas as pd
from datetime import datetime

# 1. Read source data
df = pd.read_excel("raw_sales.xlsx")

# 2. Clean - remove rows with no sales amount
df = df.dropna(subset=["Sales Amount"])
df["Date"] = pd.to_datetime(df["Date"])

# 3. Filter current month
this_month = datetime.now().month
df = df[df["Date"].dt.month == this_month]

# 4. Summarise by region
summary = df.groupby("Region").agg(
    Total_Sales=("Sales Amount", "sum"),
    Order_Count=("Order ID", "count"),
    Avg_Order=("Sales Amount", "mean")
).round(2).reset_index()

# 5. Sort by total
summary = summary.sort_values("Total_Sales", ascending=False)

# 6. Export
summary.to_excel(f"report_{datetime.now().strftime('%Y%m')}.xlsx", index=False)
print(f"Report saved: {len(summary)} regions")

Reading from SQL database with Python

import pandas as pd
import pyodbc  # or pymysql, psycopg2 depending on your DB

conn = pyodbc.connect(
    "DRIVER={SQL Server};SERVER=myserver;DATABASE=mydb;Trusted_Connection=yes"
)
query = """
SELECT region, SUM(sales) as total
FROM sales_table
WHERE MONTH(sale_date) = MONTH(GETDATE())
GROUP BY region
"""
df = pd.read_sql(query, conn)
conn.close()
df.to_excel("monthly_by_region.xlsx", index=False)

Key Pandas operations every MIS analyst needs

OperationCodeExcel equivalent
Filter rowsdf[df["Status"]=="Active"]AutoFilter
Pivot/groupdf.groupby("Region")["Sales"].sum()Pivot Table
Merge tablespd.merge(df1, df2, on="ID")VLOOKUP
Add columndf["Tax"] = df["Sales"] * 0.18Formula column
Sortdf.sort_values("Sales", ascending=False)Sort A-Z
Remove duplicatesdf.drop_duplicates(subset=["Order ID"])Remove Duplicates
Count by groupdf.groupby("Dept").size()COUNTIF

📈 Power BI — Complete Guide: Basics to Dashboards to DAX

Topic 1 — What is Power BI?

Power BI is a Business Intelligence (BI) tool developed by Microsoft. It helps you connect, transform, visualise, and share data to make better business decisions. BI means turning raw data into useful information — Power BI handles Data Collection, Data Analysis, Data Visualisation, Reporting, and Better Decision Making in one platform.

💡 Flow: Raw Data → Power BI Desktop → Power BI Service → Power BI Mobile → Useful Information

Power BI Architecture

ComponentWhat it doesNote
Power BI DesktopFree Windows app — connect, transform, model data, create reportsStart here. Download free from powerbi.microsoft.com
Power BI ServiceOnline SaaS platform — publish, share, collaborate, dashboardapp.powerbi.com — teams view reports in browser
Power BI MobileAndroid/iOS/Windows app — view reports & dashboards on the goAccess reports from anywhere

Power BI Licensing

LicenceWhat you get
Power BI FreeUse Power BI Desktop and basic sharing
Power BI ProSharing, collaboration, and larger datasets
Power BI PremiumAdvanced features, large organisations, dedicated capacity

Topic 2 — Power BI Desktop Interface

Interface AreaPurpose
RibbonContains all important commands and tools
Views (Report / Data / Model)Switch between creating reports, previewing data, and managing relationships
Report CanvasArea where you design and build your reports
Visualisations PaneSelect chart type, assign values, set filters and format
Fields PaneShows all tables and columns from your data
Page TabsAdd, delete, and manage report pages

Topic 3 & 10 — Data Sources & Get Data

Power BI can connect to a wide variety of data sources. Use Home → Get Data to connect.

CategoryExamplesDescription
FileExcel (.xlsx, .xls), CSV, Text, JSON, XMLData stored on your computer or network
DatabaseSQL Server, MySQL, PostgreSQL, Oracle, IBM DB2Data stored in relational databases
CloudAzure SQL, Azure Blob Storage, Google BigQueryData stored in cloud services
Online ServicesSharePoint List, Salesforce, Google AnalyticsData from online platforms
OtherWeb, OData Feed, Blank QueryWeb pages, APIs, custom queries

Data Import Modes — critical to understand

ModeDescriptionBest For
Import ModeData is imported and stored in Power BI. Fast performance and all features availableSmaller datasets, fast reports
DirectQueryData is NOT imported. Power BI sends queries to source in real-time. Always up-to-dateLarge datasets, real-time data from databases
Dual (Composite)Combination of Import and DirectQueryLarge models with aggregations
💡 Tip: For best performance use Import Mode. Use DirectQuery only for large real-time data. Always clean and prepare data before using in reports.

Topic 4 & 15 — Power Query Editor (ETL)

Power Query is a data preparation and transformation tool in Power BI. Use it to clean, shape and transform raw data before loading it into the data model. Access via Home → Transform Data.

OperationWhat it does
Remove ColumnsRemove unnecessary columns from data
Remove RowsRemove unwanted rows (blanks, errors, duplicates)
Split ColumnsSplit a column into multiple columns
Merge ColumnsCombine two or more columns into one
Change Data TypeChange type to Text, Number, Date etc.
Replace ValuesFind and replace specific values
Remove DuplicatesRemove duplicate rows
Merge QueriesCombine two tables based on a common column (like SQL JOIN / VLOOKUP)
Append QueriesAdd rows from one table to another (like SQL UNION ALL)
Pivot/Unpivot ColumnReshape data from rows to columns or columns to rows
Custom ColumnCreate a new column using a custom formula
💡 Applied Steps: Every transformation you apply is recorded in the Applied Steps list. Steps are executed in order from top to bottom. You can edit, delete, reorder or disable any step. Always keep steps simple, reusable and well organised.

Power Query Keyboard Shortcuts

ActionShortcut
Close & ApplyAlt + F4
Advanced EditorAlt + F11
Go to StepCtrl + G
Move Step Up/DownCtrl + Up/Down Arrow

Topic 5, 11 & 16 — Data Modeling in Power BI

Data modeling is the process of creating relationships between tables. It helps organise data and make it easier to analyse. A data model is a collection of tables, relationships, and calculations (measures and columns).

Types of Tables

Table TypeDescriptionExamples
Fact TableContains measurable data (numbers)Sales, Orders, Transactions
Dimension TableContains descriptive data (details)Customer, Product, Date, Region

Types of Relationships

TypeDescriptionExampleSymbol
One-to-One (1:1)One record in Table A relates to one record in Table BEmployee ↔ Employee Detail1 — 1
One-to-Many (1:*)One record in Table A relates to many records in Table BCustomer → Many Sales1 — *
Many-to-Many (*:*)Many records in Table A relate to many records in Table BStudent ↔ Course* — *

Cross Filter Direction

DirectionBehaviourWhen to Use
Single (Default)Filter flows from Table A to Table B onlyMost cases — better performance
BothFilter flows in both directionsOnly when you have a specific requirement

Schema Types

SchemaDescriptionRecommendation
Star SchemaFact table at center, dimension tables around it. Simple and fast✅ Recommended for Power BI
Snowflake SchemaDimension tables are normalised into multiple related tablesUse when dimension data is very large and has hierarchy

Best Practices for Data Modeling

Use a Star Schema. Keep fact tables at the center. Use meaningful and consistent names. Create relationships using unique key columns. Avoid many-to-many relationships if possible. Hide unnecessary columns from report view. Use a Date Table for time intelligence. Always relate dimensions to facts, not to each other.

Topic 6, 12 & 17 — DAX (Data Analysis Expressions)

DAX is the formula language used in Power BI to create calculations in tables or models. DAX is used to create Measures (dynamic calculations), Calculated Columns (static, stored in model), and Calculated Tables (new tables using DAX).

Basic DAX Syntax

-- Basic syntax: Result = FUNCTION( Column1, Column2, ... )
-- Example:
Total Sales = SUM(Sales[Amount])
Avg Sales = AVERAGE(Sales[Amount])
Total Orders = COUNT(Sales[OrderID])
Unique Customers = DISTINCTCOUNT(Customers[CustomerID])

Essential DAX Functions

DAX FunctionWhat it doesExample
SUM()Adds all values in a columnTotal Sales = SUM(Sales[Amount])
AVERAGE()Returns average of valuesAvg Sales = AVERAGE(Sales[Amount])
COUNT()Counts number of rowsTotal Orders = COUNT(Sales[OrderID])
DISTINCTCOUNT()Counts unique valuesUnique Customers = DISTINCTCOUNT(Customers[CustomerID])
MIN() / MAX()Returns min or max valueMin Sales = MIN(Sales[Amount])
CALCULATE()Modifies filter context for a measureSales 2024 = CALCULATE(SUM(Sales[Amount]), Sales[Year]=2024)
FILTER()Returns table that satisfies conditionFILTER(Sales, Sales[Amount] > 1000)
ALL()Removes all filters from table/columnAll Sales = CALCULATE(SUM(Sales[Amount]), ALL(Sales))
RELATED()Gets related value from another tableCity = RELATED(Customer[City])
IF()Conditional logicStatus = IF(Sales[Amount] > 1000, "High", "Low")
RANKX()Rank rows by a measureRank = RANKX(ALL(Products), [Total Sales])
SUMX()Iterate rows and sum expressionRevenue = SUMX(Orders, Orders[Qty] * Orders[Price])
COUNTROWS()Counts number of rows in a tableTotal Orders = COUNTROWS(Sales)

CALCULATE — the most important DAX function

-- CALCULATE changes the filter context
-- Example: Sales for 2024 only
Sales 2024 = CALCULATE(
    SUM(Sales[Amount]),
    Sales[Year] = 2024
)

-- Sales YTD using time intelligence
Sales YTD = CALCULATE(
    SUM(Sales[Amount]),
    DATESYTD(Date[Date])
)

Filter Context vs Row Context

ContextDescriptionExample
Filter ContextDetermines which rows are considered in a calculation. Comes from slicers, filters, visuals, and CALCULATETotal Sales = SUM(Sales[Amount])
Row ContextExists when a formula is evaluated row by row. Comes from calculated columns or iteratorsDiscounted Price = Sales[Price] * 0.9

Time Intelligence Functions

FunctionSyntaxUse
TOTALYTDTOTALYTD(Expression, DateColumn)Year to date total
SAMEPERIODLASTYEARSAMEPERIODLASTYEAR(DateColumn)Same period last year
DATESYTDDATESYTD(Date[Date])Returns dates year to date
DATEADDDATEADD(Date[Date], -1, YEAR)Shift dates by a period

Topic 18 — Measures vs Calculated Columns

AspectCalculated ColumnMeasure
Calculation TimeCalculated at data refresh timeCalculated at query time
StorageStored in the data modelNot stored — calculated on the fly
Size ImpactIncreases model sizeDoes NOT increase model size
Used InRows, Filters, Slicers, GroupsValues, KPIs, Cards, Charts
Changes with FiltersNo (static per row)Yes (dynamic)
Best ForStoring data and row-level logicAggregations and business calculations
💡 Rule of thumb: Use Columns to store data. Use Measures for calculations. The right choice = better performance and accurate reports.

Topic 7 — Power BI Visualisations

VisualBest Used For
Bar / Column ChartCompare categories across items (horizontal = Bar, vertical = Column)
Line / Area ChartShow trends over time, patterns
Pie / Donut ChartShow part of a whole (few categories only)
Scatter ChartShow relationship/correlation between two values and outliers
MapShow geographical data by country, state, city
TreemapShow hierarchical data in rectangles
Gauge / KPIShow progress towards a target/goal
Card / Multi-row CardShow single KPI values, key numbers
Table / MatrixDetailed data view, summarised data with subtotals and hierarchies
Funnel ChartShow stages in a process (conversion rates)
Waterfall ChartShow cumulative effect of positive and negative values

Which Visual to Use — Quick Guide

GoalUse
Compare categoriesBar / Column Chart
Show trend over timeLine / Area Chart
Show part of wholePie / Donut Chart
Show relationship between two valuesScatter Chart
Show geographical dataMap
Show hierarchical dataTreemap / Matrix
Show progress to goalKPI / Gauge
Show flow or stagesFunnel Chart
Show increase/decreaseWaterfall Chart

Topic 8 — Filters & Slicers in Power BI

TypeWhat it does
Visual Level FilterFilters data for a specific visual only
Page Level FilterFilters data for the entire page
Report Level FilterFilters data for all pages in the report
Drillthrough FilterPasses data to a drillthrough page for more detailed analysis

Filters vs Slicers

FeatureFiltersSlicers
PurposeRestrict data shown in visualsAllow user to interact and filter data
VisibilityUsually hidden in Filter paneVisible on the report page
User InteractionNot directly visible to usersUsers can click and select values
TypesVisual, Page, Report, DrillthroughDropdown, List, Date, Between
Best Used ForPage/Report level filtering or advanced conditionsQuick filtering and interactive analysis

Topic 9 — Formatting & Design in Power BI

Good formatting makes your report attractive, easy to read and more effective. Use the Format pane to modify each visual.

Formatting OptionDescriptionBest Practice
TitleAdd or change the title of the visualUse clear and meaningful titles
BackgroundChange background colour or add transparencyUse light colours for better look
Data ColoursChange colours of data pointsUse different colours for categories
Data LabelsShow values on chartsShow value on bars or pie slices
GridlinesShow or hide gridlinesHelps in better readability
Conditional FormattingHighlight data based on conditions (Data Bars, Color Scale, Icon Sets)Use for KPIs, targets, performance tracking

Topic 14 & 19 — Power BI Dashboards & Publishing

A dashboard is a single-page view of your most important visuals from one or more reports. It is ideal for monitoring KPIs and tracking performance. Dashboards are interactive and update with real-time data.

Dashboard vs Report

FeatureDashboardReport
PurposeMonitor KPIs and key metrics at a glanceAnalyse and explore detailed data
LayoutSingle page (tile-based)Multiple pages (detailed)
InteractivityHigh-level interactivityHigh (Filters, slicers, drill)
Data SourceCan use multiple reportsBased on a single dataset
Best ForQuick insights and monitoringIn-depth analysis and storytelling

Step-by-step: build your first Power BI dashboard

StepActionNotes
1Home → Get Data → Excel / SQL Server / CSVConnect to your data source
2Transform Data → Power Query EditorClean: remove blanks, rename columns, fix data types
3Model view → create relationshipsLink tables by common key column — like SQL JOINs
4Create DAX measuresTotal Sales = SUM(Sales[Amount]), YTD, Growth %
5Report view → add visualsBar chart, line chart, card, KPI, table, map
6Add slicers for interactivityDate slicer, region slicer — user filters on click
7Format visuals and apply themeView → Themes. Consistent colours, clear labels
8File → Publish to Power BI ServiceShare URL with team — live dashboard, no email needed

Topic 20 — Power BI Service (Publish, Share & Collaborate)

TaskHow to Do It
Pin a visual to dashboardOpen report → Focus mode → Pin (pushpin icon) → Select dashboard
Share a reportClick Share → enter email with specific permissions
Publish to webFile → Embed report → Copy link or code
Schedule data refreshDataset settings → Scheduled refresh → Set frequency (daily, hourly)

Workspaces in Power BI Service

TypeUse
My WorkspacePersonal workspace for individual use
Team WorkspaceCollaborate with a team or group
Premium WorkspaceFor datasets >1GB and advanced features
App WorkspaceDistribute content at scale with apps

Row-Level Security (RLS)

RLS allows you to restrict data access based on user roles. Users see only the data they are allowed to see. Example: North region manager sees only North data, South manager sees only South data. Set up in Model view → Manage Roles → define DAX filter rules per role.

Topic 13 — Navigation & Interactivity

FeatureDescriptionHow to Enable
Drill DownView data at lower level of detail (Category → Subcategory)Click drill icon on visual
Drill ThroughNavigate to a different page with more detailsRight-click data point → Drill through
BookmarksSave and go to a specific view of your reportView → Bookmarks → Add
Sync SlicersApply slicer selection to multiple pagesView → Sync Slicers panel
Report TooltipsShow additional info when hovering over a visualCreate a page → Page Information → Tooltip: On
Q&A VisualAsk questions in natural language and get answersAdd Q&A visual from visualisations pane

Power BI vs Excel — when to use which

ScenarioUse ExcelUse Power BI
One-off analysis for yourself
Live dashboard for management
Share with non-technical usersEmail file (risky)✅ Share URL, always fresh
Data over 1 million rows❌ Slow / crashes✅ Handles very large datasets
Quick ad-hoc calculation
Multiple data sources in one reportComplex✅ Built-in
Automated scheduled refreshManual✅ Scheduled refresh in Service
Role-based access (RLS)❌ Not available✅ Row-Level Security built-in

Sample DAX Measures Reference

-- Common measures every Power BI developer needs

Total Sales = SUM(Sales[Amount])
Total Quantity = SUM(Sales[Quantity])
Average Price = AVERAGE(Sales[Price])
Total Orders = COUNTROWS(Sales)
Unique Customers = DISTINCTCOUNT(Customers[CustomerID])

-- Profit Margin %
Profit Margin % = DIVIDE([Total Profit], [Total Sales], 0)

-- Running Total
Running Total = 
CALCULATE(
    SUM(Sales[Amount]),
    FILTER(ALL(Date), Date[Date] <= MAX(Date[Date]))
)

-- Sales vs Last Year
Sales Last Year = 
CALCULATE(
    [Total Sales],
    SAMEPERIODLASTYEAR(Date[Date])
)

-- YTD Sales
Sales YTD = 
TOTALYTD([Total Sales], Date[Date])

-- Sales 2024 only
Sales 2024 = 
CALCULATE(
    [Total Sales],
    YEAR(Date[Date]) = 2024
)

-- Top 10 Customers (Calculated Table)
Top Customers = 
TOPN(10, Customers, [Total Sales], DESC)

⚙️ Apache Airflow — Schedule and Orchestrate MIS Pipelines

What Airflow does in plain English

Airflow is a scheduler for data pipelines. Instead of running your Python script manually every Monday morning, Airflow runs it automatically at 6am, sends you an alert if it fails, and keeps a log of every run. Think of it as Windows Task Scheduler but for data teams — far more powerful and with full visibility.

Key concepts

ConceptWhat it meansReal example
DAGDirected Acyclic Graph — your pipeline as a Python filemonthly_sales_report.py defines what runs and in what order
TaskOne step in the pipelineextract_data → clean_data → generate_report → send_email
OperatorType of taskPythonOperator (run Python), BashOperator (run shell), EmailOperator (send email)
ScheduleWhen to run — cron format"0 6 * * 1" = every Monday 6am
TriggerWhat starts the DAGSchedule, manual trigger, or when another DAG finishes

Simple MIS pipeline in Airflow

from airflow import DAG
from airflow.operators.python import PythonOperator
from datetime import datetime, timedelta

def extract(): 
    # query database, save to CSV
    pass

def transform():
    # pandas cleaning and aggregation
    pass

def load():
    # write to Excel, send email
    pass

with DAG("mis_monthly_report",
         schedule_interval="0 6 1 * *",  # 6am on 1st of every month
         start_date=datetime(2025,1,1),
         catchup=False) as dag:

    t1 = PythonOperator(task_id="extract", python_callable=extract)
    t2 = PythonOperator(task_id="transform", python_callable=transform)
    t3 = PythonOperator(task_id="load", python_callable=load)

    t1 >> t2 >> t3  # defines the order

🗺️ MIS Learning Roadmap — Month by Month

MonthFocusTarget outcomeDifficulty
Month 1-2Excel masteryPivot tables, VLOOKUP, SUMIFS, basic macros, clean formattingBeginner
Month 3-4SQL fundamentalsSELECT, WHERE, JOIN, GROUP BY — query any database independentlyBeginner-Medium
Month 5-6Python basics + PandasRead/write Excel, filter/group/merge data, automate one real taskMedium
Month 7-8Power BIBuild a live dashboard connected to SQL or Excel, share with teamMedium
Month 9-10Python advanced + SQL advancedETL pipelines, scheduled reports, complex SQL (CTEs, window functions)Medium-High
Month 11-12Airflow + Cloud basicsAutomated pipelines on a schedule, Azure Data Factory or AWS Glue basicsHigh
💡 Practical tip: After each month, build one real project using that skill at work. Theory without practice is forgotten in 2 weeks. The goal is not to finish the roadmap — it is to make your work 20% faster each month.

🎯 MIS Interview Questions

MIS · BEGINNER
What is a VLOOKUP and when would you use it?
VLOOKUP = Vertical Lookup. It searches for a value in the leftmost column of a table and returns a value in the same row from a specified column. Syntax: =VLOOKUP(lookup_value, table_array, col_index_num, [range_lookup]). Use FALSE for exact match. When to use: joining data from two sheets by a common ID. Example: you have employee IDs in Sheet1 and a salary table in Sheet2. =VLOOKUP(A2,Sheet2!A:C,3,FALSE) returns the salary. Limitation: VLOOKUP can only look to the RIGHT. If you need to look left, use INDEX/MATCH instead.
MIS · BEGINNER
What is the difference between COUNT, COUNTA, and COUNTIF in Excel?
COUNT counts only numeric cells. COUNTA counts ALL non-empty cells including text. COUNTIF counts cells that match a specific condition. Examples: COUNT(A:A) — how many cells in column A have numbers. COUNTA(A:A) — how many cells are not empty. COUNTIF(B:B,"Completed") — how many cells in column B say "Completed". COUNTIFS extends this to multiple conditions: COUNTIFS(Region,"North",Status,"Active") = count rows where both conditions are true.
MIS · ENGINEER
A manager asks for a report showing top 10 products by revenue this quarter. Walk through how you would build this in SQL.
SELECT product_name, SUM(revenue) as total_revenue, COUNT(order_id) as order_count FROM orders WHERE order_date >= DATEADD(quarter, DATEDIFF(quarter,0,GETDATE()),0) AND order_date < GETDATE() GROUP BY product_name ORDER BY total_revenue DESC LIMIT 10 (or TOP 10 in SQL Server before the SELECT). Walk through: SELECT the product name and aggregate revenue and order count. FROM the orders table. WHERE filter to current quarter — DATEDIFF/DATEADD calculates the first day of current quarter. GROUP BY product so aggregates are per product. ORDER BY DESC to get highest first. LIMIT/TOP 10 for only the top 10. In an interview, always explain WHY each clause is there, not just what it does.
MIS · ENGINEER
You have a 500MB Excel file that takes 10 minutes to open and crashes regularly. What do you do?
This is a Python/SQL problem, not an Excel problem. Step 1 — identify what is making it large: formulas that reference entire columns (=SUM(A:A) instead of =SUM(A2:A10000)), images or charts embedded, many pivot caches, no filters applied to limit data. Step 2 — migrate to Python: import pandas as pd; df = pd.read_excel("large.xlsx"). Pandas handles 500MB easily. Step 3 — if data comes from a database, connect Python directly to the DB with pd.read_sql() instead of exporting to Excel first. Step 4 — output only the summary, not raw data, back to Excel: summary.to_excel("report.xlsx"). The original 500MB becomes a 10KB summary report.
MIS · ARCHITECT
Design a monthly sales report automation for 50 regional managers. Data is in SQL Server, output is email with Excel attachment.
Architecture: Python script triggered by Airflow on the 1st of every month at 6am. Step 1 — Python reads regional manager list from SQL: SELECT manager_id, name, email, region FROM managers. Step 2 — Loop through each manager. Step 3 — For each manager, run SQL query filtered to their region: SELECT product, SUM(sales) FROM orders WHERE region = ? AND MONTH(date) = last_month GROUP BY product. Step 4 — Use Pandas + openpyxl to create a formatted Excel file per manager with their data only. Step 5 — Use smtplib or SendGrid to email each manager their personalised report. Airflow tracks success/failure per manager. If one email fails, only that one retries — other 49 already sent. Total time for 50 reports: under 2 minutes. Without automation: one person spending a full day each month.