Wizard of Legend

There's no shortcut for this; you'll just have to grind out runs until the chaos gem counter is maxed out at 999. This won't take as long if you unlock the "Graduation Cap" relic and keep it equipped. Each Council Member and end-of-floor fight is guaranteed to drop a few, and you will also occasionally get them during normal fights.

You can easily get between 50-85 gems each run! Each run for me so far has been about 6-8 minutes.

So what you need to do to get lots of gems each run is to have the TOKEN OF WEALTH relic and the Avarice outfit.

You can find the TOKEN OF WEALTH relic in the area with all of the shops; it will be to the left under the red and white tarp. If you don't see the Token of Wealth on the table, travel back to the room where you always spawn in and travel back to the shops, and those items on the table will be different.

You can find the Avarice outfit in the top right corner of the shop district under the green and tan tarp (If you don't see it there, you might need to buy one of the outfits to refresh it).

The Token of Wealth relic allows you to have a banker spawn in on the Chaos Trials which will convert your gold into chaos gems.
The Avarice outfit increases your gold gain by 15%, and your chaos gem gain by 1.

So what you want to do is have the ToW relic active and the Avarice outfit on.

Now start up a chaos trial and clear out all the enemies around the map and destroy most of the breakables (barrels, benches, chairs, etc.). Once you find the banker (he'll be in a room of gold, and a red portal. It should only be him in the room), if you have cleared out all the areas and have around 200-300+ gold, interact with the banker and convert your gold to gems; you should have gotten around 55+ chaos gems.
Now all you want to do is kill the boss at the end of stage 1-1 (the room with the big black circle on the ground and possibly the portals). After you kill him, you'll get another 10 gems after destroying the big chest. Now just go find a pit and keep dashing into the pit until you die. Rinse and repeat!

Update: you can kill the boss, then go to the banker if you want. At first, I thought the banker disappears, but he's still there after you kill the boss!

If something needs clarification or something I can add to the guide, let me know in the comments!

Graduation Cap with the Avarice robe makes this a breeze. You get +2 gems on every drop. This also seems to work with Jade's Gem Locator, meaning you regularly (and randomly) find 3 gems just lying around the map. I was moving pretty recklessly and still got 150+ gems per run in just the first few floors with sub-20 minutes runs.