What's normal behavior?
To see the changes, we need to establish the baseline. There are many behaviors that are fundamental to what fish do and don't normally change very much at all. Many fish like to live in groups, both in the wild and in captivity. Corydoras catfish, for example, live in schools comprising hundreds of individuals, and in captivity they are certainly a lot more outgoing and settled when kept in groups than singly. Sociable fish often have hierarchies that define the status of each member within the group. Extreme examples of this can be found among the marine Clownfishes, where the biggest fish in the group is the female and all the smaller individuals males. This isn't because the male and female fish come together to form such a group, but because all baby Clownfishes are males, and only if one of them becomes the biggest and most aggressive fish in the group does it change into a female.
Even where fishes don't change sex, there will often be a definite pecking order in the group. From our perspective, it isn't always which fish is in charge, but watching the fish sparring with each other as they establish and maintain that hierarchy is easy enough. Tiger barbs chase one another, tetras flick their fins, Firemouth cichlids flare their gill covers, and halfbeaks wrestle. But as well as having ways to pick fights, schooling fish need to be able to stop fights too, not least of all because the survival of depends on all its members being equally alert and agile. Often, the weaker fish displays a signal that makes it clear that it accepts the higher status of the bigger fish within the group. Many cichlids use color for this, such as dark vertical bands on the flanks that advertise to aggressive males quite clearly that the submissive individual is not a threat. Many of the schooling, open water cichlids of Lake Malawi show this very well, as do discus and angelfish.
Of course, not all fishes live and many are intensely territorial, driving away other members of their own species from their home range. A lot of damselfish, Bettas, cichlids, and Loricariid (Suckermouth) catfish are animals of this type. Territoriality serves a variety of functions. In the case of damselfish, the territory-holding fish try to monopolize Gardens of algae that they cultivate as a dependable source of food. Male Bettas and cichlids, on the other hand, are guarding nesting sites that they will use to attract females and raise their young.
Schooling behavior especially seems to go wrong in aquaria under certain circumstances. Where fish get along peacefully in groups of six or twelve when kept in twos or threes they can often be very aggressive to one another. Monos, archerfish, angelfish, and piranhas are among fish species that can be very problematical when kept in too small a group. It seems to be the hierarchy that fails by not being spread out hiwayhifionline.com a large enough number of fish, with the result that the dominant fish ends up bullying the one or two other fish in the group, often to the point of death by preventing them from feeding properly. With other species, this aggression spills over into the rest of the tank, and the fish become nippy and waspish towards their tankmates. Tiger barbs are the classic examples of this, being fin-nippers when kept in small groups but usually much more peaceful when kept in a group of a dozen or more.
Something that fishes in aquaria have to deal with that they don't in the wild are the full range of interactions with other species of fish. Wild fish will have to compete with rival species for food or nesting sites, and small species will need to avoid being eaten by larger ones, but within the aquarium, these relationships are taken to a whole new level because of the lack of space and high stocking density. Where dwarf cichlids like kribs may have several square feet of territory to work with in the wild, in an aquarium the same area may be home to several pairs of fish, not to mention assorted catfish, loaches, and other bottom-dwelling species.
Corydoras catfish in particular seem to have a very bad time with dwarf cichlids. In the wild, Corydoras live in creeks and streams, typically moving across the riverbed as a large group, replying on their schooling behaviour and stout body armor to keep them safe from predators. Dwarf cichlids, and indeed most other small, territory-holding fish, tend to live in sluggish waters, particularly pools and riverbanks where the caves and burrows are more abundant. The poor Corydoras catfishes just doesn't seem to be able to get their heads around the concepts of territories and boundaries, and blithely swim into patches held by dwarf cichlids and end up getting nipped and chased as a result. In a small aquarium, mixing Corydoras even with cichlids as otherwise benign as Apistogramma and Pelvicachromis can be very unwise. Loaches and Loricariids on the other hand not only understand territories, but hold territories of their own as well. As a result, both tend to work rather well with cichlids of comparable size, quickly establishing an armed truce where each fish respects the boundaries set by the other.
Midwater fish will also have to find ways of getting along with species they would never encounter in the wild. In my own community tank I have some glassfish and silver hatchetfish, the former from Asia and the latter from South America. Strangely enough, they can and do interact. Perhaps their shape and coloration is similar enough that they treat each other as equals? Certainly their mode of establishing a hierarchy Ã' best described as dive-bombing one another Ã' appears to translate pretty well, and the two species will alternate between short chases and schooling freely apparently smoothly and without harm to either party.
Sometimes though, there are gaps in communication that make it difficult or impossible for the two species to communicate with one another. An example of this occurred in a slightly brackish aquarium I kept many years ago, within which were some sailfin mollies and a group of large Melanotaenia rainbowfish. Rainbows are from Australia and mollies from the Americas, so again, this is a combination that wouldn't occur naturally. The problem was the high, arched back of the rainbowfish was too similar to a male sailfin molly with its dorsal fin erect. The male mollies would swim around the rainbows, raising their fins, and generally being aggressive and annoying. Male mollies can of course lower their dorsal fins when they're done fighting, but the poor rainbowfish couldn't do anything to diffuse the situation but swim away, and this only worked until a rainbow and a male molly bumped into each other again.
On the other hand, I've seen combinations where inter-species communication doesn't just fail it; makes things worse. Colombian shark catfish, Hexanematichthys seemanni, thrive in marine tanks, and on once occasion I put a trio of these catfish into a large marine aquarium alongside a variety of species including a small blue triggerfish, Odonus niger. Blue triggers are amicable enough (for triggerfish, anyway) and Colombian sharks are entirely peaceful, so on paper at least this looked good. In reality, it was a disaster. The problem was communication: both species produce drumming and clicking sounds, but they do so for entirely different reasons. Shark catfish produce sounds to help the school stick together in murky water and also to warn potential predators that they are venomous and best left alone (like the rattlesnakes rattle). Triggers use sound as a threat. As soon as the catfish started making noise, the triggerfish immediately went on the offensive, and started badgering the catfish. The more threatened the catfish were, the more noisy they became, and so the triggerfish became even more aggressive. Needless to say, the two species ended up being separated.