The Markov property indicates that, with knowledge of the current state, previous trajectories are irrelevant for predicting the probability of the future of a process. A Markov chain is a discrete-time stochastic (i.e. random) process possessing the Markov property. Probabilities and expected values on a Markov chain can be evaluated by a technique called First Step Analysis. An analogous technique can be applied to continuous-time processes. We will discuss an elementary introduction to Markov chains and First Step Analysis, followed by a broader description and discussion of the long-term behavior of Markov chains. Further, we will get acquainted with the Poisson Processes which are continuous-time processes with finite number of states, and, finally, will overview the continuous processes and their applications.