Project Documentation & SDLC

Understanding project documentation is crucial for data analytics professionals.

Whether you’re building BI solutions, data pipelines, or analytics platforms, proper documentation ensures alignment between business needs and technical implementation.

Why Project Documentation Matters

You might think filling out documents is just “bike-shedding” or a waste of time. However, when working cross-functionally with several teams, proper documentation is essential for tracking the stream of value and resource utilization.

The Documentation Hierarchy

Artifact	Question / Purpose	Primary Owner
Project Charter	”Go/No-Go”: Do we authorize this project?	Project Sponsor
BRD	”Why are we doing this from a business perspective?”	Business Analyst
PRD	”What product features will solve the problem?”	Product Manager
FRD	”How will the system specifically behave?”	Business/Systems Analyst
Project Estimation	”How much time and resources needed?”	Project Manager
RACI Matrix	”Who is responsible for what?”	Project Manager

The Project Charter

What is a Project Charter?

The Project Charter is the foundational document that comes before all others. Think of it as the project’s “birth certificate.”

Key Characteristics:

Formally authorizes the project’s existence
Empowers the Project Manager to apply organizational resources
Provides high-level overview of purpose, objectives, and scope
Aligns stakeholders on goals and boundaries
Serves as a reference point throughout the project lifecycle

Who Defines It?

The Project Manager drafts it in collaboration with:

Project Sponsor (provides financial resources and support)
Key Stakeholders

The Project Sponsor’s formal approval and signature are essential.

What It Contains

Essential Components:

Project Purpose: Why does this project exist?
Business Objectives: What will success look like?
High-Level Scope: What’s included/excluded?
Key Deliverables: What will be produced?
High-Level Timeline: When will it be done?
High-Level Budget: How much will it cost?
Major Stakeholders: Who’s involved?
High-Level Risks: What could go wrong?

When It’s Created

Project Initiation Phase (before detailed planning):

Business Case/Feasibility Study (often precedes charter)
Project Charter ← Created here
Then: BRD, PRD, FRD, detailed planning

Analogy:

The Project Charter is like getting permission and initial funding to build a house, agreeing on its general size and purpose (e.g., “We’re building a three-bedroom family home”).

The Business Requirements Document (BRD)

Understanding the BRD

Think of the BRD as the project’s “constitution” - it defines the fundamental purpose and essential outcomes.

The BRD as a Constitution:

Defines the “Why”: Core values and reasons for existence
Sets the “What”: Fundamental capabilities and functions needed
Establishes the “Who”: Key stakeholders and their expectations
Source of Truth: Primary reference for business aspects
Living Document: Reviewed and updated as understanding evolves
For Policy Makers: Business leaders, stakeholders, end-users

Questions a BRD Must Answer

1. What is the Problem or Opportunity?

What business pain point are we addressing?
What market opportunity are we pursuing?
Why is this project necessary now?

2. What are the Business Objectives?

What measurable goals will this achieve?
How will success be defined and measured?
Examples: “Increase customer retention by 15%”, “Reduce costs by 10%”

3. What is the Scope (Business Perspective)?

What business functions are included?
What business functions are excluded?
What high-level features are needed?

4. Who are the Stakeholders?

Who are the primary users/beneficiaries?
What are their core requirements?
What are their expectations?

5. What are the Key Business Processes?

How does the business currently operate?
How will processes change/improve?
What workflows need to be understood?

6. What are the High-Level Requirements?

What must the solution do to meet objectives?
Are there regulatory/compliance needs?
What are the performance expectations?

7. What are the Assumptions and Constraints?

What assumptions about resources, technology, timelines?
What known limitations must we work within?

8. What are the Deliverables and Success Metrics?

What tangible outputs are expected?
How will business measure success?

BRD Best Practices

Key Principles:

Engage stakeholders early to gather input
Avoid technical jargon for accessibility
Use visuals (diagrams, flowcharts, tables)
Be concise while providing necessary detail
Prioritize requirements by business impact
Keep it living - review and update regularly

Analogy:

If the Project Charter says “We’re building a three-bedroom family home,” the BRD explains: “Our growing family needs more space, and we want it to be energy-efficient.”

The Product Requirements Document (PRD)

Understanding the PRD

The PRD translates business needs into specific product features and user experiences.

Who Defines It:

The Product Manager or Product Owner, collaborating with:

Designers
Development leads
Business stakeholders

Purpose:

Define specific features and functionalities
Describe user experiences
Act as the voice of the user and market
Bridge business needs and technical implementation

What It Contains

Typical Sections:

Product Vision: Overall goal and purpose
User Personas: Who will use this?
User Stories: What can users do?
Feature Specifications: Detailed feature descriptions
User Flows: How users navigate the product
Success Metrics: How to measure product success
Prioritization: What’s critical vs nice-to-have

Analogy:

The PRD details what features the house will have: “Open-concept living area, modern kitchen with island, two-car garage.”

Practical Example: User Story with Acceptance Criteria

User stories are a key component of the PRD. They bridge business needs (BRD) and technical specifications (FRD).

The Story Format:

As a [type of user],
I want to [perform some action]
So that [achieve some goal/benefit]

Example: User Authentication

User Story: Account Access

As a Registered User,
I want to log into the application using my email and password
So that I can access my private dashboard and saved data securely.

Acceptance Criteria (Functional Logic):

These criteria define what “done” looks like and serve as the foundation for the FRD:

UI Presentation: User is presented with a clean login screen containing Email and Password fields
Input Validation: The “Login” button is disabled if the fields are empty
Authentication: System checks the database to verify user exists and password matches
Success Path: If credentials are correct, user is redirected to /dashboard
Error Handling: If credentials fail, show generic error: “Invalid email or password” (do not specify which one for security)
Session Management: User session is maintained (JWT or Session cookie) so they don’t have to re-login on refresh
Security: Password field characters must be hidden by default (masked input)

Why This Matters:

This user story demonstrates:

Clear user perspective: Focuses on who and why
Testable criteria: Each checkbox can be verified
Security considerations: Generic error messages, password masking
Technical hints: Mentions JWT/Session cookies, redirects

From User Story to FRD:

The FRD would then expand these acceptance criteria into detailed technical specifications:

Functional Requirement FR-AUTH-001: User Login

1. Input Fields:
   - Email: Text input, max 255 characters, email format validation
   - Password: Password input (type="password"), max 128 characters

2. Validation Logic:
   - Client-side: Check fields not empty before enabling button
   - Server-side: Query users table WHERE email = input_email
   - Password verification: Use bcrypt.compare(input_password, stored_hash)

3. Success Response:
   - HTTP 200 status
   - Generate JWT token with 24-hour expiration
   - Set httpOnly cookie with token
   - Redirect to /dashboard

4. Error Response:
   - HTTP 401 status
   - Return: {"error": "Invalid email or password"}
   - Do NOT specify which field is incorrect
   - Log failed attempt with timestamp and IP

5. Session Persistence:
   - Store JWT in httpOnly, secure cookie
   - Validate token on each protected route request
   - Refresh token if < 2 hours until expiration

Key Takeaway:

User stories (PRD level) focus on what the user needs.

Acceptance criteria provide testable conditions.

The FRD then specifies exactly how the system will implement it.

SMART vs INVEST: Writing Better Requirements

Two complementary frameworks help you write effective requirements:

INVEST is for User Stories (PRD level)
SMART is for Acceptance Criteria and Objectives (FRD level)

INVEST Framework (For User Stories)

INVEST ensures your user stories are well-formed and actionable:

Independent: Can be developed without heavy dependencies on other stories
Negotiable: Details can be discussed and refined
Valuable: Delivers clear value to users or business
Estimable: Team can estimate the effort required
Small: Can be completed within a single sprint
Testable: Has clear criteria to verify completion

Example: Good vs Bad User Stories

❌ Bad (Violates INVEST):

As a user, I want the system to be fast and secure.

Problems:

Not Independent (affects everything)
Not Negotiable (too vague)
Not Estimable (what does “fast” mean?)
Not Small (too broad)
Not Testable (no clear criteria)

✅ Good (Follows INVEST):

As a data analyst,
I want to export dashboard data to CSV format
So that I can perform additional analysis in Excel.

Why it works:

✅ Independent: Standalone feature
✅ Negotiable: Can discuss CSV format, file size limits
✅ Valuable: Enables offline analysis
✅ Estimable: Team can estimate development time
✅ Small: Can complete in one sprint
✅ Testable: Can verify CSV export works

SMART Framework (For Acceptance Criteria)

SMART ensures your acceptance criteria and objectives are clear and measurable:

Specific: Clearly defined, no ambiguity
Measurable: Can track progress and completion
Achievable: Realistic and attainable
Relevant: Aligned with overall goals
Time-bound: Has a defined timeframe or trigger

Example: Applying SMART to Acceptance Criteria

User Story (INVEST):

As a sales manager,
I want to receive alerts when regional sales drop below target
So that I can take immediate corrective action.

Acceptance Criteria (SMART):

✅ SMART Criteria 1:

Given I am a sales manager with active alerts enabled,
When any region's daily sales fall below 90% of the daily target,
Then I receive an email notification within 15 minutes,
And the email includes the region name, actual sales, target sales, and variance percentage.

Why it’s SMART:

Specific: Exactly what triggers the alert (90% threshold)
Measurable: Can verify 15-minute delivery, email content
Achievable: Technically feasible with current systems
Relevant: Directly supports the user’s goal
Time-bound: 15-minute SLA

✅ SMART Criteria 2:

Given I receive a sales alert,
When I click the "View Details" link in the email,
Then I am redirected to the regional dashboard showing the last 7 days of sales data,
And the underperforming region is highlighted in red.

Why it’s SMART:

Specific: Clear action (click link) and result (dashboard view)
Measurable: Can verify redirect works, 7-day data shown
Achievable: Uses existing dashboard infrastructure
Relevant: Enables quick investigation
Time-bound: Immediate redirect (< 2 seconds)

❌ Bad (Not SMART):

The system should notify users about problems.

Problems:

Not Specific (what problems? which users?)
Not Measurable (how to verify?)
Not Time-bound (when should notification happen?)

Complete Example: INVEST + SMART Together

Scenario: Building a data quality monitoring feature

1. User Story (INVEST):

As a data engineer,
I want to be notified when data pipeline failures occur
So that I can fix issues before they impact downstream reports.

INVEST Check:

✅ Independent: Standalone notification system
✅ Negotiable: Can discuss notification channels (email, Slack, SMS)
✅ Valuable: Prevents report delays, saves time
✅ Estimable: Team estimates 3-5 days
✅ Small: Fits in one sprint
✅ Testable: Can simulate failures and verify notifications

2. Acceptance Criteria (SMART):

AC1: Pipeline Failure Detection

Given a data pipeline is running,
When the pipeline fails at any stage (extraction, transformation, loading),
Then the system logs the failure with timestamp, stage, and error message,
And triggers a notification within 5 minutes.

SMART Analysis:

Specific: Defines what constitutes failure (any stage)
Measurable: 5-minute SLA, logged details
Achievable: Standard error handling
Relevant: Core requirement for monitoring
Time-bound: 5-minute notification window

AC2: Notification Delivery

Given a pipeline failure has been detected,
When the notification is triggered,
Then an email is sent to the data engineering team distribution list,
And the email subject contains "[URGENT] Pipeline Failure: [Pipeline Name]",
And the email body includes: pipeline name, failure time, stage, error message, and link to logs.

SMART Analysis:

Specific: Exact email format and content
Measurable: Can verify email received with correct content
Achievable: Standard email integration
Relevant: Provides actionable information
Time-bound: Immediate upon detection

AC3: Failure Resolution Tracking

Given I receive a pipeline failure notification,
When I click "View in Dashboard" link,
Then I am redirected to the pipeline monitoring dashboard,
And the failed pipeline is displayed at the top with status "Failed",
And I can see the last 5 runs with their status (Success/Failed),
And I can download the error logs as a .txt file.

SMART Analysis:

Specific: Clear dashboard requirements
Measurable: Can verify all elements present
Achievable: Uses existing dashboard framework
Relevant: Enables quick troubleshooting
Time-bound: Immediate redirect

AC4: Success Confirmation

Given a pipeline has failed and been fixed,
When the pipeline runs successfully,
Then a confirmation email is sent with subject "[RESOLVED] Pipeline Restored: [Pipeline Name]",
And the failure is marked as "Resolved" in the dashboard,
And the resolution time is logged for reporting.

SMART Analysis:

Specific: Defines success state and confirmation
Measurable: Can verify resolution email and logging
Achievable: Standard workflow
Relevant: Closes the loop on incident
Time-bound: Immediate upon successful run

Key Differences: INVEST vs SMART

Aspect	INVEST (User Stories)	SMART (Acceptance Criteria)
Purpose	Define user needs	Define completion criteria
Level	High-level feature	Detailed requirements
Audience	Product team, stakeholders	Developers, QA testers
Focus	What and why	How to verify
Format	Narrative (As a… I want…)	Given-When-Then
Flexibility	Negotiable details	Specific and measurable

Best Practices

For User Stories (INVEST):

Write from user’s perspective
Focus on value, not implementation
Keep it conversational
One story = one feature
Include the “so that” (the why)

For Acceptance Criteria (SMART):

Use Given-When-Then format
Be specific about inputs and outputs
Include edge cases and error scenarios
Make them independently testable
Avoid technical implementation details

Common Mistakes:

❌ Mixing levels:

User Story: "The system shall validate email format using regex"

This is too technical for a user story - it’s an acceptance criterion.

✅ Correct separation:

User Story: "As a user, I want to register with my email 
            so that I can receive account notifications"

Acceptance Criteria: "Given I enter an email address,
                      When I click Register,
                      Then the system validates the email format,
                      And rejects invalid formats with message 'Please enter a valid email'"

How They Work Together

graph TD
    A[Business Need] --> B[User Story - INVEST]
    B --> C[Acceptance Criteria - SMART]
    C --> D[Functional Requirements - FRD]
    D --> E[Implementation]
    
    style B fill:#e1f5ff
    style C fill:#fff4e1
    style D fill:#ffe1e1

The Flow:

INVEST helps you write a good user story (PRD level)
SMART helps you write testable acceptance criteria
FRD expands criteria into technical specifications
Development implements based on FRD
Testing validates against SMART criteria

The Functional Requirements Document (FRD)

Understanding the FRD

If the BRD is the “What” and “Why,” the FRD is the “How” from a system perspective.

The FRD as an Instruction Manual:

Think of it as the detailed instruction manual for the system’s machinery:

The “Buttons and Levers”: Every specific function and interaction
The “Rules of Operation”: Precise logic and behavior
The “Blueprint for Engineers”: Exact specifications for developers
The “Test Script Baseline”: Foundation for QA testing
The “Granular Detail”: Drills down from broad strokes to specifics

Questions an FRD Must Answer

1. What exactly will the system do?

Specific features and functionalities
Detailed steps for each user interaction
Example: “User clicks X, System displays Y, User enters Z”

2. How will the system respond to inputs?

Exact inputs the system accepts
Precise output for each input
Validation rules and error handling

3. What business rules must the system enforce?

Calculations and validations
Logic and algorithms
Examples: “10% discount on orders over $500”, “Password must be 8+ characters”

4. How will data be handled?

Data inputs and processing
Data storage and attributes
Data manipulation and presentation

5. How will the system integrate with others?

Integration points with existing systems
Data exchange formats and protocols
API specifications

6. What are the non-functional requirements?

Performance:

Response time requirements
Example: “Page load under 2 seconds for 90% of requests”

Security:

Encryption requirements
Access control specifications
Example: “Role-based access control must be enforced”

Usability:

UI/UX requirements
Example: “Navigation menu visible on all pages”

Reliability:

Uptime requirements
Example: “System uptime must be 99.9%”

Scalability:

Concurrent user support
Transaction volume handling

7. What are error handling procedures?

Error logging and notification
Recovery procedures
User-friendly error messages

FRD Best Practices

Key Principles:

Be specific and measurable: Avoid ambiguity
Use clear language: Technical but understandable
Include examples: Show expected behavior
Trace to BRD: Every FRD requirement links to BRD
Consider testability: Each requirement must be verifiable

Analogy:

The FRD specifies: “The kitchen island will have a built-in sink with specific plumbing requirements, and the garage door will be controlled via an app.”

BRD vs FRD vs PRD

Key Distinctions

Aspect	BRD	PRD	FRD
Focus	Business needs	Product features	System behavior
Audience	Business stakeholders	Product team	Developers, QA
Level	High-level	Medium-level	Detailed
Language	Business terms	User-centric	Technical
Question	”Why?” and “What?"	"What features?"	"How exactly?”

The Relationship

graph TD
    A[Project Charter] --> B[BRD: Business Requirements]
    B --> C[PRD: Product Requirements]
    C --> D[FRD: Functional Requirements]
    D --> E[Technical Design]
    E --> F[Development]

Flow Example:

Project Charter: “Build a new online store to increase sales by 20%”
BRD: “The store needs product browsing, cart, checkout, payment methods, and order tracking”
PRD: “Users can search products, add to cart, checkout in 3 steps, receive email notifications”
FRD: “Search shall support keyword filtering. Checkout shall integrate with PayPal API. Email notifications sent via SendGrid with specific templates”

The SDLC Connection

Where Documentation Fits in SDLC

1. Initiation Phase:

Business Case/Feasibility Study
Project Charter ← Created here

2. Requirements & Analysis Phase:

BRD ← Primary output
PRD ← Defines product features
FRD ← Created late in this phase

3. Design Phase:

FRD is primary input
System architecture designed
Database structure planned
UI/UX designs created

4. Development Phase:

FRD serves as specification
Developers code to FRD requirements
Minimizes assumptions and rework

5. Testing Phase:

FRD drives test cases
QA verifies each functional requirement
BRD guides UAT (User Acceptance Testing)

6. Deployment & Maintenance:

Documents serve as reference
Historical record for future enhancements

Interdependence and Evolution

Key Points:

Documents are interdependent and iterative
BRD comes first, setting high-level vision
FRD elaborates on BRD, translating to system specifics
Changes in BRD necessitate FRD changes
Technical feasibility may adjust BRD

In Agile:

Documents are lighter and more iterative
User stories replace detailed FRDs
BRD may be captured in product backlog
Continuous refinement throughout sprints

Project Estimation

When Estimation Happens

Estimation is an ongoing, iterative process that becomes more refined over time.

1. Initiation Phase (Very Early):

Type: Rough Order of Magnitude (ROM)

Accuracy: -25% to +75%
Purpose: Determine feasibility, secure initial budget
Who: Senior management, sponsors, experienced PMs
Based on: Similar past projects, industry benchmarks

2. Planning Phase (After BRD, during PRD/FRD):

Type: Definitive estimates

Accuracy: -10% to +25% (improving to -5% to +10%)
Purpose: Detailed budget, schedule, resource plan
Who:
- Project Manager (facilitates)
- Product Manager (clarifies features)
- Development Team (provides task estimates)
- QA Team (estimates testing)
- Other teams (design, marketing, etc.)
Techniques: Analogous, parametric, bottom-up estimating

3. Execution Phase (Throughout Development):

Type: Refined and updated estimates

Process: Progressive elaboration
Purpose: Track progress, manage costs, forecast completion
Who: PM with development team and stakeholders

Estimation Best Practices

Key Principles:

Involve the doers: People doing the work provide best estimates
Break down work: Smaller tasks are easier to estimate
Use historical data: Learn from past projects
Include buffers: Account for unknowns
Update regularly: Refine as you learn more

AI and Vibe Coding Context

Why This Matters for AI

Proper documentation is crucial when working with AI coding assistants:

1. Better Initial Prompts:

BRD-style thinking helps you articulate business needs clearly
FRD-style specificity gives AI precise requirements
Results in better first iterations

2. Context for LLMs:

Structured requirements help AI understand full context
Reduces back-and-forth clarifications
Improves code quality and alignment

3. Validation:

Clear requirements make it easier to verify AI output
FRD serves as test criteria for generated code

Example: AI-Ready Requirements

Poor Prompt:

Build a dashboard for sales data

BRD-Informed Prompt:

Business Need: Sales team needs real-time visibility into 
regional performance to identify underperforming areas and 
take corrective action within 24 hours.

Success Criteria: Dashboard must show sales by region, 
compare to targets, highlight regions >10% below target.

FRD-Informed Prompt:

System Requirements:
1. Dashboard shall refresh every 5 minutes
2. Display sales data aggregated by region (North, South, East, West)
3. Show actual vs target with variance percentage
4. Highlight in red any region with variance < -10%
5. Include drill-down to see top 5 products per region
6. Export to PDF functionality
7. Role-based access: Sales Managers see all regions, 
   Regional Managers see only their region

Best Practices

Do’s

✅ Start with Project Charter before detailed requirements ✅ Engage stakeholders early in BRD creation ✅ Keep documents living - update as you learn ✅ Trace requirements from BRD → PRD → FRD ✅ Use visuals to clarify complex requirements ✅ Prioritize ruthlessly - not everything is critical ✅ Make it testable - every requirement should be verifiable ✅ Version control your documentation

Don’ts

❌ Skip the charter and jump to coding ❌ Write FRD without BRD - you’ll miss the “why” ❌ Use jargon in BRD - keep it business-friendly ❌ Make it static - requirements evolve ❌ Forget non-functional requirements (performance, security) ❌ Ignore stakeholder input ❌ Over-document - be concise and relevant ❌ Lose traceability between documents

Conclusion

Proper project documentation is not bureaucracy - it’s the foundation for successful delivery:

Key Takeaways:

Project Charter authorizes and empowers
BRD defines business needs and why
PRD specifies product features and user experience
FRD details system behavior and how
All documents work together through SDLC
Estimation evolves from rough to refined
Good documentation enables better AI assistance

Remember:

Documents are communication tools, not just paperwork
They align teams and prevent costly misunderstandings
They’re especially valuable in cross-functional projects
They make AI coding assistants more effective

Next Steps:

Understand your role in the documentation process
Learn to write clear, testable requirements
Practice translating business needs to technical specs
Use documentation to improve AI prompts
Keep documents updated as projects evolve

The time invested in proper documentation pays dividends throughout the project lifecycle and beyond.