Away3D has Awayphysics library that is alchemy port of C++ engine and show very good performance. Away3D has clean readable source code and documentation + good set of example projects. It has 3ds max plugin that can export scene and generate code to render it. Support popular formats like 3ds, obj... and You can write importer plugin for own format. All so developers are pretty cool guys that answer questions on forum :)
For JavaScript the most popular engine is three.js Look what it can do.
General thoughts:
WebGL support isn't good at this time(sometimes it depends not only on user browser but all so on drivers) and You will need to deal with cross browser headache. Flash isn't supported on IOS and get kicked more and more from device browsers. But there is cross-platform adobe air for standalone apps.
Javascript, in my opinion, isn't good for writing such projects and its debugging can cause permanent brain damage. WebGL shader language is GLSL that has nice C-like syntax. Ordinary Flash shaders are written on assembler but You may try pixelbender3d. At this time Flash shaders more limited than WebGL(for example you can't fetch texture in vertex shader) but it's done to make Flash more compatible and it's not a problem for most cases. WebGL is tightly integrated with page.
If you want java WebGL and like pain try GWT 3D engines or gwt-elemental and there is tree-js port at early stage.
x3dom is declarative WebGL 3D engine with fallback to Flash it is ez but pretty limited and unstable.