Introduction

Protection mechanisms in operating systems define methods to safeguard resources from unauthorized access, modification, or denial of service. They enforce security policies, maintain system integrity, and ensure user privacy by controlling interactions between processes, users, and hardware components.

"Security is not a product, but a process." -- Bruce Schneier

Access Control

Definition

Access control restricts resource use by verifying permissions tied to identities or roles. Core to system security. Types: discretionary, mandatory, role-based.

Discretionary Access Control (DAC)

Owner-based control. Permissions assigned by resource owner. Flexibility: high. Risk: user errors, weak policy enforcement.

Mandatory Access Control (MAC)

System-enforced rules. Labels security levels to subjects and objects. Used in high-security environments. No user override.

Role-Based Access Control (RBAC)

Permissions assigned to roles, users inherit roles. Simplifies management in large systems. Enforces least privilege.

Authentication

Purpose

Verify user or process identity before granting access. Forms: knowledge, possession, inherence.

Methods

Password-based: secret knowledge. Biometric: physical traits. Token-based: possession of devices or keys.

Multi-factor Authentication

Combines two or more methods. Increases security. Common in sensitive systems.

Authorization

Definition

Determines what authenticated users/processes can do. Enforces policies on resource access and operations.

Access Control Lists (ACLs)

Lists associating permissions with users/groups for each resource. Efficient for fine-grained control.

Capability Lists

Tokens or keys granting access rights. Held by users/processes. Prevents unauthorized delegation.

Security Policies

Definition

Formal rules defining allowed and forbidden actions. Basis for protection mechanisms.

Types

Confidentiality, integrity, availability policies. Examples: Bell-LaPadula (confidentiality), Biba (integrity).

Policy Enforcement

Implemented by OS kernel, security modules, middleware. Continuous monitoring essential.

Memory Protection

Purpose

Prevent unauthorized access or modification of memory regions. Crucial for process isolation and stability.

Mechanisms

Base and limit registers: define valid address ranges. Paging and segmentation: hardware-assisted control.

Protection Faults

Triggered on illegal access. OS handles via exceptions or signals. Enables detection of malicious behavior.

Memory Protection MechanismDescription
Base and Limit RegistersDefines address range accessible to a process. Hardware enforced.
PagingMemory divided into fixed-size pages; access controlled via page tables.
SegmentationMemory divided into variable-size segments with protection bits.

Rings of Protection

Concept

Hierarchical privilege levels. Inner rings: higher privilege. Outer rings: less privilege.

Implementation

Commonly 4 rings (0-3). Ring 0: kernel mode. Rings 1-3: user modes, drivers, applications.

Benefits

Limits damage from compromised code. Enforces separation of concerns.

Ring 0: Kernel Mode - Full access to hardware and memory.Ring 1: Device Drivers - Limited access.Ring 2: System Utilities - Restricted access.Ring 3: User Applications - Minimal privileges. 

Capability Lists

Definition

Data structures listing access rights (capabilities) held by subjects. Tokens that prove access authorization.

Advantages

Easy delegation. Fine-grained control. Reduces need for global lookup.

Challenges

Revocation complexity. Requires secure storage and transmission.

Encryption Techniques

Role in Protection

Protects data confidentiality and integrity. Used for secure communication and storage.

Symmetric Encryption

Single key for encryption and decryption. Fast. Requires secure key distribution.

Asymmetric Encryption

Public-private key pairs. Enables secure key exchange and digital signatures.

Encryption TypeCharacteristicsUse Cases
SymmetricFast, single shared keyDisk encryption, VPNs
AsymmetricSlower, key pairsKey exchange, email security

Sandboxing

Definition

Isolates executing code in controlled environment. Limits access to system resources.

Techniques

Virtual machines, containers, language-based sandboxes.

Applications

Testing untrusted code, running plugins, mitigating malware.

Intrusion Detection

Purpose

Monitor system for unauthorized or malicious activity. Alerts administrators or triggers response.

Types

Signature-based: known attack patterns. Anomaly-based: deviation from baseline behavior.

Integration

Embedded in OS or deployed as separate modules. Requires continuous updates and tuning.

Secure OS Design

Principles

Least privilege, defense in depth, fail-safe defaults, auditability.

Examples

SELinux, TrustedBSD, Windows NT security architecture.

Challenges

Balancing usability and security. Managing complexity in policies and enforcement.

Secure OS Design Principles:- Least Privilege: minimal necessary access.- Defense in Depth: multiple layers of protection.- Fail-Safe Defaults: deny access unless explicitly allowed.- Auditing: logging for accountability. 

References

  • D.E. Denning, "A Lattice Model of Secure Information Flow," Communications of the ACM, vol. 19, no. 5, 1976, pp. 236-243.
  • R.S. Sandhu, "Role-Based Access Control," Advances in Computers, vol. 46, 1998, pp. 237-286.
  • G.J. Simmons, "Controlling Access to Information," Proceedings of the IEEE, vol. 76, no. 11, 1988, pp. 1427-1442.
  • R. Anderson, "Security Engineering: A Guide to Building Dependable Distributed Systems," Wiley, 2020, pp. 112-135.
  • L. Gong, "Inside Java 2 Platform Security: Architecture, API Design, and Implementation," Addison-Wesley, 1999, pp. 50-75.