Self-organized Femtocells: a Time Difference Learning Approach

The use model of mobile networks has drastically changed in recent years. Next generation devices and new applications have made the availability of high quality wireless data everywhere a necessity for mobile users. Thus, cellular networks must be highly improved in terms of coverage and capacity. Networks that include smart entities and functionalities, and that allow to fulfil all the mobile networks? new requirements are called heterogeneous networks. One key component in heterogeneous networks is femtocells. Femtocells are low range, low power mobile base stations deployed by the end consumers, which underlay the macrocell system and provide a solution to the problem of indoor coverage for mobile communications. Femtocells can reuse the radio spectrum and, thereby, they allow increasing the spectral efficiency. Moreover, under appropriate algorithms for interference control, they give a viable alternative to the problem of spectrum static allocation. In the case of femtocells reusing the spectrum, it must be guaranteed that the interference they generate does not affect the performance of the macrocellular system they underlay. To this end, we propose to model the femtocell network as a decentralized system and to introduce a learning algorithm in each femto node, providing self-organization capabilities. We thus introduce a multiagent learning algorithm that performs the radio resource management in the femtocell system, so that femtocells control the interference they generate in a decentralized and uncoordinated manner. Femtocells, then, are able to maintain their interference under a desirable threshold as a function of the environmental situation they perceive. In distributed systems, learning can be a long process. For this reason, we introduce a new cooperative method, known as docitive algorithm, where agents exchange information that allows them to accelerate their learning process and increase its precision. We also present a learning technique based on Fuzzy Inference Systems, which allows to represent the environment perceived by the agents and the actions they can perform in a continuous way. In this way, agents have more accurate behaviors and better adapt to the environmental conditions. Besides, we extend the learning method to partial observable environments in order to provide a 3GPP standard compliant solution, which does not rely on the existence of an X2 interface between macro and femto nodes. Finally, we present a study regarding the implementation requirements, in terms of computation and memory demands, in order to determine if the proposed solutions fit in state of the art communication processors.