Here is the short definition: Ludology is the discipline that studies games. As I see it, ludology studies games and playing in general, leaving videogames a just a particular branch of study.
The first time I used this term was in my paper “Ludology meets narratology”. I argued that one of the reasons most researchers try to use narratology to explain videogames is that there is a lack of a formal discipline that focus on games. I argued that videogames are not new construction that one day suddenly appeared on Earth, but they heavily draw upon traditional games (among other cultural artifacts, such as comics, film, television, sports, etc). Thus, in order to understand videogame, we first need to understand games. Sadly, there is very little work on this discipline (some classic authors include Caillois, Huizinga, Piaget).
Ludology is constructed upon the latin word “ludus” (game). The term has historically been used to describe the study of games and particularly of board games. I would not necessarily include “game theory (a branch of applied mathematics) as part of ludology, mainly because it usually requires some ideal situations that are better suited for applications in fields as economy and decision-making rather than traditional games. But, again, “game theory” is not a single theory, but a group of them.
As I understand it, ludology includes videogame theory, but it goes beyond it to include all games and forms of play. The current state of videogame research is mainly driven by scholars who try to explain computer games through previously existing media. For example, Brenda Laurel’s work is based on drama, Janet Murray’s on storytelling, drama and narrative, and Lev Manovich's on film. While I do not necessarily discard these approaches, I think that they are incomplete and that by studying videogames as something else than games, they are denying its main potential. This potential is not narrative, but simulation: the ability to represent dynamic systems. A picture of a dog represents a particular dog: we can learn about its shape, color, etc. A simulated dog as Sony’s Aibo or Mindscape’s Dogz is not only made through signs but also through rules of behavior. In order to understand Aibo we do not only interpret its signs, but we also must experiment with it in order to be able to infer some of its behavioral rules. To make a long story short, representation is about signs, while simulation is about signs and behavior. This is the ontological difference that makes me claim that games cannot be understood through theories derived from narrative.
Last, but not least, the goal of ludology is to understand games and videogames, both in a formal way and also as part of the media ecology. While a formalist approach to games will surely be rejected by academia (it's not in vogue anymore because postmortemism tells us that everything is anything ;), I will still take that risk. A formalist approach is needed, at least during the first years of ludogical development, in order to understand that particularities of games and videogames. Sure, we will have plenty of time later to discard these concepts and replace them with a more "modern" theory.