perf: optimize httpparser with C++23 string_view, ispanstream, and transparent maps#203
perf: optimize httpparser with C++23 string_view, ispanstream, and transparent maps#203clauspruefer with Copilot wants to merge 4 commits into
Conversation
…arent maps
- Bump C++ standard from c++20 to c++23 (enables <spanstream>)
- httpparser.hpp:
- Add StringHash with is_transparent for zero-copy heterogeneous map lookup
- Replace typedef with using for RequestHeader_t, URLParamMap_t (now use StringHash + equal_to<>)
- Add non-destructive split(string_view, string_view, vector<string_view>&) overload
- Destructive split(string&, ...) kept for _HTTPRequestBuffer management
- rsplit: change first param string& -> string_view (avoids copy at call site)
- is_digits: change param const string& -> string_view
- _parseRequestProperties: change string& -> string_view
- _parseRequestHeaders: change string& -> string_view
- _parseGETParameter: change const string& -> string_view
- getRequests(): return const RequestsMap_t& instead of value copy
- httpparser.cpp:
- #include <spanstream>
- _parseRequestHeaders: replace vector<string>/split/rsplit loop with ispanstream
line-by-line parse; no intermediate string vector; uses string_view key/value slices
- _parseGETParameter: URLParamsPart now string_view (no heap copy of URL suffix);
uses non-destructive split into vector<string_view>; removes manual push_back
- Replace replace(0, N, "") with erase(0, N) in two POST payload sites
- Remove redundant .substr(0, length()) in header value emplace
- Use const auto& (not auto) for string_view loop variable in _parseGETParameter - Add clarifying safety comments to rsplit underflow guards
clauspruefer
left a comment
There was a problem hiding this comment.
@copilot Looks good on the first sight. But please add the following tests and functionality: a) performance test / comparison to old parsing behaviour, use apx. 500 random generated valid HTTP requests containing different header counts / payload sizes, repeat the tests 500 times and save the results (including comparison data) as CSV b) do the same as in performance tests (a) but measure the used memory and also save the results as CSV
- test/performance/http-parser/test-parser-performance.cpp: 500 random HTTP requests × 500 iterations; times new (ispanstream / string_view) vs legacy (vector<string> / rsplit) header and GET-param parsing; writes results-performance.csv with per-request avg/min/max ns and speedup columns. Observed 3.3x average header-parse speedup. - test/performance/http-parser/test-parser-memory.cpp: Same 500 requests; overrides global operator new to count gross bytes allocated during each parse; writes results-memory.csv with per-request alloc count / bytes and reduction-% columns. Observed 72% byte reduction for header parsing. - test/performance/http-parser/CMakeLists.txt: build targets for both standalone executables (no Boost dependency). - test/performance/CMakeLists.txt: add_subdirectory(http-parser).
- Use chrono::nanoseconds::rep for Nanos type alias - Replace C-style sizeof casts with static_cast<int> - Add thread-safety comment to global tracking variables - Fix median() for even-length vectors (average two central elements)
Added in commit
Both executables are standalone (no Boost dependency) and are wired into the CMake build via |
The HTTP parser made unnecessary heap allocations throughout parsing — destructive string splits, full copies for URL substrings, intermediate
vector<string>for header lines, and map lookups that constructed temporarystd::stringkeys.C++ standard
-std=c++20→-std=c++23inCMakeLists.txtto unlock<spanstream>httpparser.hppStringHashwithis_transparent;RequestHeader_tandURLParamMap_tnow useunordered_map<string, string, StringHash, equal_to<>>—.find()/.at()on string literals no longer constructs a temporarystd::stringsplitoverload:split(string_view, string_view, vector<string_view>&)slices into the original buffer with zero per-token heap allocation; the existing destructivesplit(string&, …)is retained for_HTTPRequestBuffermanagementrsplit/is_digits: Parameters narrowed fromstring&/const string&tostring_view_parseRequestProperties,_parseRequestHeaders,_parseGETParameterall acceptstring_viewinstead ofstring&/const string&getRequests(): Returnsconst RequestsMap_t&instead of a by-value copyhttpparser.cpp_parseRequestHeaders: Replaced the destructivesplit+rsplitloop (which allocated avector<string>and mutated the request string in place) withstd::ispanstreamline-by-line parsing; key and value arestring_viewslices — the only allocations are the finalemplaceinto the map_parseGETParameter:URLParamsPartis now astring_viewinto the URL (nosubstrcopy); uses the new non-destructivesplitintovector<string_view>eraseoverreplace: Replacedreplace(0, N, "")witherase(0, N)at both POST payload consumption sites