Here's the problem; you keep on assuming that good tools will magically appear, that help with debugging. But good tools take a lot of time and work to perfect. In reality, you usually wind up with just barely good enough tools.
With a text based protocol, you can inspect it visually with no special tools, and munge it with general purpose tools that you already know how to use (shell script, sed, awk, perl, python, ruby, what have you) with no special support libraries or anything of the sort. Support libraries can help you with the more complex aspects of the protocols, but for basic debugging purposes, you can do it all with general purpose tools.
With a binary protocol, you need those libraries to even have a chance of being able to work with it. Now you can't use a general purpose shell pipeline to munge it; no more nc | grep or what have you. You have to have a wireshark dissector; and good luck figuring out how to grep through the results of what a wireshark dissector generates.
The main point is that the overhead of the ASCII encoding isn't the main problem with HTTP. Reading ASCII encoded CRLF delimited headers is a solved problem (and heck, you could probably switch that to just LF delimiters, since I'm sure that most processors already handle that case just fine).
The problems are things like having to repeat headers over and over again for each request in a session, enormous cookies that need to get sent with every request, and the like. But you can solve those without throwing away the easily debuggable ASCII-encoded headers; and compression really does solve most of the problem with the inefficiency of ASCII encoding (and you're going to want to use anyhow, since the HTML, CSS, and JavaScript that you're delivering is all a fairly inefficient ASCII representation too).
With a text based protocol, you can inspect it visually with no special tools, and munge it with general purpose tools that you already know how to use (shell script, sed, awk, perl, python, ruby, what have you) with no special support libraries or anything of the sort. Support libraries can help you with the more complex aspects of the protocols, but for basic debugging purposes, you can do it all with general purpose tools.
With a binary protocol, you need those libraries to even have a chance of being able to work with it. Now you can't use a general purpose shell pipeline to munge it; no more nc | grep or what have you. You have to have a wireshark dissector; and good luck figuring out how to grep through the results of what a wireshark dissector generates.
The main point is that the overhead of the ASCII encoding isn't the main problem with HTTP. Reading ASCII encoded CRLF delimited headers is a solved problem (and heck, you could probably switch that to just LF delimiters, since I'm sure that most processors already handle that case just fine).
The problems are things like having to repeat headers over and over again for each request in a session, enormous cookies that need to get sent with every request, and the like. But you can solve those without throwing away the easily debuggable ASCII-encoded headers; and compression really does solve most of the problem with the inefficiency of ASCII encoding (and you're going to want to use anyhow, since the HTML, CSS, and JavaScript that you're delivering is all a fairly inefficient ASCII representation too).