Software Programming Basics

With almost 20 years of experience of delivering Java J2EE development projects, I realize that there are several peers who have dropped out with the perception that IT is complicated and the ones who are there are unable to handle the ever changing nature of this field.

I lean on the very basic fundamental concepts to create or validate a software solution or troubleshoot existing software or programming issues.

Without elaborating much on the problems I would get down to the basics first, which I believe are enough for a programmer, analyst and executives to validate IT related issues.

Software Development or Software Programming or Coding in my opinion has the following basic fundamental concepts:

1 - Business Entities / Classes and Methods / Domain Objects - These are pieces of code that are used to encapsulate certain attributes of a Business entity and Methods encapsulate logic to perform operations on the attributes.

2 - Logical Blocks - These are If Else constructs that help in adding decision making capabilities to a Method's logic or code.

3- File Input Output / File Read Write operations.

4- Database Input Output / File Read Write operations - These are not as straightforward as using File IO API/methods calls but are generally done by SQL queries.

5- Multi Threading - These are Classes and Methods to execute multiple operations at the same time.

6- Socket Programming - These are APIs used to connect two machines to each other. All machine to machine communication happen using Socket Programming. TCP, HTTP, FTP are protocols meaning a set of Strings or instructions that each side of the machines understand.

7- GUI Programming - These are UI development APIs that help develop screens and webpages that we interact with.

8- Collection classes, Data Structures, Algorithms. I have covered a snapshot of Java collection classes in a blog.
https://sourabhchat-java.blogspot.com/2017/01/java-collections-framework-snapshot.html

9- Application Layering/ J2EE Design patterns - This is to organize the logical blocks of a application and a web application so that all of the code is not cluttered into one class but is divided into category of classes each doing a specific function. I have published a separate blog on this topic.

Application Layering based on basic J2EE design patterns
https://sourabhchat-java.blogspot.com/2017/01/j2ee-design-patterns-and-variants.html

Application Layering based on Advanced J2EE Design Patterns and complex architectures
http://sourabhchat-architecture.blogspot.com/


10-Gangs of four design patterns - These are standard ways to organize your Classes so that it is easy to understand for designers and code maintenance. I have published a very simplified blog on this. Any corrections on this blog are welcome.

https://sourabhchat-java.blogspot.com/2017/06/core-java-design-patterns-gangs-of-four.html


Some generic programming terms:
Function/method is a single block of code or subroutine to perform some tasks. Library or Utility or 3rd party Jars are a set of bundle of classes for reuse usually to achieve some specific set of generic features like pdf writer or our comms library.. API is application programming interface..this is similar or same as the libraries..we have jdbc api or security package.. but also nowadays APIs are generally referred to as the online internet based webservices that can be exposed to public for consumption of refrence data like forex conversion APIs or data submit APIs.. framework is a set of layers of classes to restrict the way you design your program ..frameworks generally standardize the way developers write ui controller services dao classes.

Any business problem in my opinion can be resolved using a combination of the above and and issue can be analyzed by asking the right questions using the above.

Of course, experience plays a vital role, but even for established developers the above should be a backup plan to resolve development issues using a well established and stable language and APIs.

Today a lot of the new generation developers do have the natural capability to use the above logic concepts and they are voracious learners but at the same time they somehow seem like parrots repeating the benefits of new languages, products and APIs without realizing the immense costs of adoption of new Products and end up using technologies for not so well suited use cases or by considering all possible use cases.

Keeping in mind the above list of basic building blocks that a standard programming language offers, any of the new age products could be understood at least conceptually to begin with, and this is true especially for managers.


Architectural Components 

Webservers and Application Servers are a product of Socket Programs, Multithreading, Implementing protocols like HTTP, Session Management (A Hashmap Of Hashmap like structures that maintain user data till it is saved to a Database), Implementation of a file system to deploy files as per internet specifications like J2EE.

Messaging Products like JMS, Kafka etc are basically a mishmash of Collections, Socket programs, Threads and Collections/Data structure algorithms.

No SQL databases are about having no SQL queries but having a product based on Socket programming, Accepting Client connections through Client API classes, Storing data in a cache i.e. a Data Structure/Collection and maintaining algorithms to persist data directly to a custom file system either synchronously or asynchronously.

Products like Elastic Search and Solr basically cache data in memory, ofcourse by applying specialized algorithms for efficient data fetch performance.

Products like Kibana and Grafana are used for visualization i.e. to view data in a Database table or view - in a cached grid based webpage and by configuring graphs. However these are at basic levels for now and are configuration intensive.

Big data architectures are achieved by having a Kafka like component to absorb/ ingest data at a rapid pace - or in a non blocking way - meaning not like a blocking request response manner as in http based communication. The data from Kafka is then processed in a cluster and the cluster is increased to solve the problem of Back Pressure. Back Pressure problem happens when the incoming data is huge and the cluster is not able to process as fast as the data is coming in, hence Reactive programming to scale up the servers to process increased loads is part of Big Data architectures.
To achieve scalability, please see the notes below.

Scalability is achieved using technologies like Kubernettes wherein a Server component activates a predefined set of VMs or servers as and when required and deploys the Dockers image at runtime thereby achieving scalability when the loads on the existing servers increase.

CICD Pipeline - Continuous Integration Continuous Deployments are about product stacks like Containers like Dockers that maintain the list of installers with versions so that you can move from one environment to another i.e. from Dev to Stage to Prod with the same set of product stack.

Products like Zookeeper, Ansible maintain a list of configurations per environment.

Jenkins build products are used to run ANT or Maven builds and deploy to the selected environment.

Jenkins build tools are also configured to run JUnits test cases so that any new code deployed is tested as soon as it is built enabling a validated build on the environment.

So.. this is actually a huge topic and writing endlessly would seem like redoing a engineering book, but my motivation or rather burning desire was to help several hapless engineers and programmers who are finding it difficult to cope up with the ever changing nature of the IT product stack. I have just tried to simplify and organize a step by step approach for anyone who could use this content. The truth is that if you have your basics strong based on the 10 points mentioned above, it would be very clear that the ever changing nature of IT is not so difficult to cope up with as it is only a new flavor or a product that provides better features and is built using the above 10 concepts of programming.