My Jugaad Approach to Learn System Design Thinking
I love writing a startling heading.
Jugaad is the gutsy art of overcoming harsh constraints to solve a problem using limited resources.- Harvard Business Review
My constraints are a lack of a formal degree in Computer Science. My most limited resource is time. My problem is a bunch of unknown unknowns in software.
I've been a bit misguided in treating my ignorance in software architecture independently from my deep insecurity in my programming skills. System Design Thinking is something that lets me take a simultaneous stab at programming and software architecture. I'm totally punching above my weight by setting this gargantuan goal of understanding both.
questions to ask WHiLE DESIGNING A SYSTEM
My current understanding is hardly three questions deep.
What are the business requirements? What are the minimum viable features in the product?
How do I represent the structural and behavioural features of this product through a class diagram? (Object-oriented concepts)
What service APIs will be used for this product? Third-party and internal services? What will call it and how?
What is the availability of the system? If a host or a data centre goes down, what will be the recourse?
What is the latency and how can the performance be improved? (Caching, Use CDN, HTTP/2, Reduce JS files etc.)
Will the system scale to n or more users?
How to securely store data in the database without losing it? Backups?
What is the level of security and privacy to be provided?
How can we make the system cost-effective?
Btw, expert developers reading this might find it either lacking or irrelevant to the subject. After all, what is a part of system design is entirely dependent on whether you think it is important.
To answer these questions I will require sufficient knowledge on the following non-exhaustive list of topics I've been gathering through marathon Stackoverflow browsing. I will be adding links to each of these topics as and when I find good ones on the web. Please do suggest links that you enjoyed or benefitted from reading in the comments section.
I hope to teach myself these topics and tools.
DATABASE CONCEPTS
Vertical vs horizontal scaling
CAP theorem
ACID vs BASE
Partitioning/ Sharding
Consistent Hashing
Optimistic vs Pessimistic locking
Strong vs eventual consistency
Relational Databases vs Non-relational SQL
Types of NoSQL
Key value
Wide column
Document-based
Graph-based
MEMORY CONCEPTS
Caching
Data center/ Racks/ Hosts
CPU/ Memory/ Hard drives/ Network bandwidth
Random vs sequential read/ writes to disk
NETWORK CONCEPTS
HTTP vs http2 vs WebSocket
TCP/ IP model
ipv4 vs ipv6
TCP vs UDP
Domain Name Server lookup
Load Balancer L4 & L7
Content Delivery Networks (CDNs) & Edges
SECURITY CONCEPTS
HTTP & TLS
Public key infrastructure and certificate authority(CA)
Symmetric vs asymmetric encryption
OTHER CONCEPTS
Paxos
Leader election
Design patterns and Object-oriented design
Virtual machines and containers
Publisher Subscriber over a queue architecture
MapReduce for distributed and parallel processing of big data
Multithreading, locks, synchronization, concurrency, compare-and-swap
Data Structures & Algorithms relevant to the system
Probabilistic Data Structures: Bloom filters and Count-Min sketch for big data
Tools
Cassandra
MongoDB/Couchbase
Mysql
Memcached
Redis
Zookeeper
Kafka
NGINX
HAProxy
Solr, Elastic search
Amazon S3
Docker, Kubernetes, Mesos
Hadoop/Spark and HDFS
Edit: An awesome colleague pointed me to this link: https://github.com/donnemartin/system-design-primer