After Nintendo's two biggest competitors outfitted their fifth-generation consoles with CD drives and Sony enticed many big-name companies to develop games for them, Nintendo made the controversial decision to stick with cartridges for the Nintendo 64. They also opted for 64-bit design architecture, which many developers of the time were not familiar with; the end result was a system that was notoriously difficult to develop games for and had a serious lack of third party support. Which also meant it had very few RPGs, as they are among the most difficult and time-consuming game genres to develop. Still, they did manage to get a few in there, and while they didn't have as many prominent ones as the PlayStation (or even arguably the Saturn), there were a few that are quite high in quality. And of course a few that aren't.
Best: The Legend of Zelda: Majora's Mask (Nintendo, 2000)
It may be a controversial pick, but Majora's Mask is easily the better of the two Zelda games on the N64 in my book, as well as one of my favorites in the entire series. Ocarina of Time gets credit for being the first to make the leap to 3D, but also felt constrained by the format rather than taking full advantage of it - the drab, empty overworld, the really lackluster sidequests and many of the items and dungeons are seemingly built around the limitations of early 3D rather than playing to its strengths. Well, with the release of a 4 MB RAM expansion, Majora's Mask proved that the franchise could work just as well in 3D as it could in 2D. The gameplay is now vastly expanded with Link having a total of four forms to explore the world in, some amazingly creative new dungeon puzzles and gimmicks (flip the entire dungeon upside-down!) and a creative setup where you continually replay the same three days, with each and every character operating on a timed schedule and allowing you to complete numerous side-objectives as long as you can find and complete them in time. It even takes the darker depiction of Hyrule seen in OOT to new heights, with a downright bizarre world to explore and some truly nightmarish, surreal imagery as the world comes closer and closer to its end. It's a game whose mechanics definitely take some getting used to, but it's very fun, unique and rewarding once you do.
Worst: Quest 64 (Imagineer, 1998)
Pretty much a no-brainer choice here; everybody knows Quest 64 sucks. It met with a lot of hype before its launch for being the first turn-based RPG on the platform, especially in light of how huge the genre had gotten on the PlayStation (and to a lesser extent, the Saturn). But when it launched, it quickly showed that it had little to offer any discerning genre fan. From the threadbare story to the bland protagonist, the boring use of elemental spell effects (throw bigger and bigger rocks or waterspouts or icicles for fireballs, oo!) and lacking basic features like, y'know, shops and upgradable equipment, it was just an underwhelming title all-around. The bland combat and irritating encounter rate didn't help much either - while it seems to have a bit of a tactical bent at a glance with its gridless movement and varying attack ranges for spells, it mostly just comes down to "spam rocks and heal with water spells until everything dies". Quest 64 is at least relatively bug-free and has some pretty impressive graphics for the platform, with colorful environments, detailed textures and surprisingly little of the infamous "N64 fog", but doing the bare minimum technically isn't good enough, especially in an era of unprecedented innovation for the genre.
Runner-Up: Hybrid Heaven (Konami Computer Entertainment Osaka, 1999)
Created by Konami's Osaka branch as a rival game to Metal Gear Solid (yes, really), Hybrid Heaven is a relatively unique blend of RPG elements and action-platforming reminiscent of Tomb Raider. Between fights you crawl under doors, climb up onto ledges, dodge and destroy traps and solve switch puzzles. In battle, things shift to something like Final Fantasy's Active Time Battles crossed with a fighting game - a bar steadily fills, and once it does you can unleash punches, kicks, throws, armbars, et cetera. Enemy moves are countered in similar fashion, giving you the option to try to dodge, counter or step back to avoid them entirely. It sounds like a deep system, but it's honestly not as much as you'd hope - everything is determined by dice rolls, the result of which are entirely random; there doesn't seem to be any hard counter to any given type of attack or any real benefit to using a lot of your moves. In fact, you can mostly just stick with basic jabs and kicks and go for head-strikes, which do a lot of damage and frequently knock down your opponent, allowing for several free followup hits. Enemies also tend to take very large leaps in power without warning, which can easily send you to the game over screen and force you to spend a substantial amount of time grinding weaker enemies to power up. The story is an interesting one involving aliens, clones and hybrid fighting machines, but the overall monotonous design and surprisingly slow combat really drags it down.