Well, the fact that there has not been a Jade Cocoon 3, nor any attempt by Genki to revisit the monster breeder RPG genre, probably answers that opening question for you. But let's take a closer look at the game and see exactly where this franchise met its downfall.
One of the first things the player is bound to notice is that the overall aesthetic of the game bears almost no resemblance to the original. Katsuya Kondo is no longer in the role of character designer or animation director, resulting in a drastic aesthetic overhaul. The setting of the game is no longer that of a lush, natural environment, but now has the player hopping between dimensions, going from one surreal alien locale to the next. The tone of the original game is lost as well - Jade Cocoon 2 takes on a much more irreverent, almost campy feel to its narrative, with silly story elements, over-the-top acting and outlandish character redesigns at every turn. Many characters return from the first game, though they bear little resemblance to their old counterparts; even Levant, the protagonist of Jade Cocoon, returns as what I can only describe as a "space hippie", sporting an exaggerated colorful outfit, a yin-yang pendant, an enormous purple lute and mile-long hair bangs. While never atrociously annoying, and some elements of this can be amusing - Nico's irreverent narration of the storyline easily being the highlight - it's a very stark contrast to the heavy, somber mood of the first Jade Cocoon game.
Secondly, the overall style of gameplay is heavily overhauled from the first Jade Cocoon. Instead of an RPG with pre-rendered backgrounds and linear progression, Jade Cocoon 2 is overhauled into more of a dungeon crawler. The player travels about lengthy dungeons, entering rooms in search of a spore pod which will allow them access to the next floor. Said rooms can also contain random items, monsters or trainers - either mundane ones who provide a modest fight or "Rivals" who provide much more of a challenge. Defeating other trainers earn you medals, which in turn unlock new challenges that can grant the player rare monster eggs.
Jade Cocoon 2 also has a more generic and utilitarian RPG setup overall, with the player being encouraged to undertake a number of fetch quests and defeat-the-monster challenges. These in turn grant a variety of rewards, such as rare items or monster eggs, and all will boost the player's Reputation, certain levels of which are required to take advancement exams in order to be able to unlock and use more monsters simultaneously in battle.
Which leads us to Jade Cocoon's most distinct difference: its combat system. While the first game was a standard turn-based affair not dissimilar to its main inspiration of Pokemon, Jade Cocoon 2 takes things in a far different direction, attempting to add more of a strategic bent. In essence, combat now operates with both participants having a 3x3 grid, with the protagonist and his rival in the center space and the surrounding spaces populated by up to eight monsters. The row facing the opponent is the one that deals out attacks and takes damage whenever the player takes their turn, and the shoulder buttons rotate the grid to face a different set of up to three monsters toward the opponent.
What determines the winner of a fight is not so much the monsters themselves, however, but taking care to balance attacking, defense, and not leave the player character open to attack. If the center space on the row facing the opponent is vacant, the opponent can then attack the other character directly, depleting one of their Shields; once all of their Shields are depleted, the fight ends immediately. Thus, a significant part of strategy in combat is rotating one's monsters at the right times to tank opponents' attacks, heal their group of monsters, or go on the offensive. Placing one's monsters strategically is important as well - each side of the grid is aligned with one of four elements (Earth, Wind, Water and Fire) and placing monsters that match that alignment will grant a bonus to their attacks and abilities. Monsters cannot be placed in the center space of the grid that opposes their particular element, and they will always use an attack correlating to the element of that side of the grid if they have one. Essentially, it combines elements of Ogre Battle into the formula of the first Jade Cocoon and ends up being a much more complex experience on the whole.
What it all boils down to is that Jade Cocoon 2's big failing wasn't being a bad game; it's a competently designed experience in almost every respect with some interesting mechanics and an enjoyably strange atmosphere. The problem was that it made the same mistake Chrono Cross did only a year prior, billing itself as a direct sequel despite bearing little resemblance to the first game in terms of direction, and in fact actively going out of its way to upend many established elements of the first game's lore and characterizations. Its irreverent tone, heavier emphasis on mundane quests and dungeon crawling, and a larger random element all contribute to it being a much different experience than the original. Those that were drawn in by the first game's appeal as an interesting Pokemon alternative felt left out, and the overall design and atmosphere was too bizarre to draw in fans of older and more traditional RPGs, or even ones who jumped on the bandwagon with recent hits like Final Fantasy VII that sold themselves on having a heavier tone and more adult themes. Basically, it simply failed to find much of an audience, selling about 100,000 copies to its predecessor's 270,000, and in an industry driven by revenue, that's the worst sin a franchise can commit in a publisher's eyes.
Developer: Genki
Publisher: Ubisoft
Platform: Playstation 2
Released: 2001
Recommended version: N/A