Queueing and Storage Control Models

by Sheu, Ru-Shuo

This thesis is divided into two parts. The first part is about the control of a special queueing network which has two service nodes in tandem on each service channel. With capacity at each service node being finite, we compare som different control policies to find the admission and routing policies that minimise the blocking rate in the queueing system. We obtain limit theorems as the number of channels becomes large. The stochastic optimization technique we apply here is the Lagrangian method, using the Complementary Slackness Conditions to choose the optimal action.

In the second part we consider two reservoir control problems. In the first, the cost function is a single simple linear function, and the second has two different cost functions and the choice of them forms a finite-state Markov chain. We find the optimal policies to determine how many units of water should be released from the reservoir under these two different models. We model the reservoir as a Markov decision process. The policy-iteration algorithm and the value-iteration algorithm are the main methods we apply in this part.

In both problems we apply stochastic optimization techniques. The reservoir model uses a standard Markov decision process model, with the associated methods of policy-iteration and value-iteration to find the optimal state-dependant policy. In the routing problem we also interested in state-dependent policies, but here we wish to look at the system in the limit as the number of queues becomes large, so we can no longer us the technique of Markov decision processes. We look, instead, at the limiting deterministic problem to find the optimal policy.