allinurl/goaccess
1
Fork 0
mirror of https://github.com/allinurl/goaccess.git synced 2025-06-18 22:45:36 -04:00
Code Issues Actions Packages Projects Releases Wiki Activity
4,117 Commits 3 Branches 58 Tags 12 MiB
multithread-parsing
Commit Graph

4117 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Gerardo O
70ca3677b1 Update man page to reflect multithreading parsing changes. 2024-01-03 19:31:17 -06:00
Gerardo O
d2a62bdec8 Updated man page to clarify --exclude-ip. 2024-01-03 19:22:30 -06:00
Gerardo O
e432c01d9e Updated --chunk-size help section for clarity. 2024-01-03 19:19:46 -06:00
Gerardo O
1280072fb7 Removed unnecessary comment. 2024-01-03 18:54:07 -06:00
Gerardo O
8495d4db15 Fixed warning: comparison of integer expressions with different signedness. 2024-01-03 18:22:45 -06:00
Gerardo O
49fde9680b Ensure the WebSocket server is spun up once log-format checks have passed. 
Prevents resource waste (leaks and unnecessary allocations) when handling
invalid log/date/formats.
 2024-01-03 07:55:34 -06:00
Gerardo O
9a1ad615c6 Ensure we show format errors if max tests is reached on stdin. 2024-01-02 19:41:56 -06:00
Gerardo O
c6efef6d3a Updated help section, threads and chunk-size options. 2024-01-01 09:21:20 -06:00
Gerardo O
496f9c7f7f Eliminated temporary debugging messages from the parsing code. 2023-12-31 17:55:11 -06:00
Gerardo O
8f570c85db Refactored read_lines() for improved cohesion by decomposing it into smaller functions. 2023-12-31 17:52:38 -06:00
Gerardo O
728fd7d481 Fixed issue where it will segfault when using ignore* options. 
This problem emerged after the implementation of the multithread feature in
commit 1e116d0. Additionally, it involves refactoring some of the code to align
more closely with the processing methodology used prior to the introduction of
multithread parsing.
 2023-12-31 13:32:35 -06:00
Gerardo O
0550432bea Fixed regression from commit 1e116d04 preventing stdin data piping. 2023-12-22 11:26:45 -06:00
Gerardo O
399837df3f Updated code formatting to align with the coding style. 2023-12-20 17:13:09 -06:00
Gerardo O
41e63dc1bf Ensure we don't process more than we should when testing for log format. 2023-12-20 17:10:34 -06:00
Gerardo O
1e116d04f6
Merge pull request #2594 from Sea-n/multithread-part1 
Add Multi-thread Support
 2023-12-20 17:09:21 -06:00
Sean Wei
f4ca02d405 Update .gitignore 2023-12-20 01:54:25 +08:00
Sean Wei
1a072765f0 Add multi-thread to README 2023-12-20 01:54:25 +08:00
Sean Wei
6d2cb7efae Change GLogItem->status from char* to int 
In process_log(), use array lookup instead of string compare
This will speedup process_log() by x1.38, and have x1.28 overall speedup
 2023-12-20 01:54:25 +08:00
Sean Wei
4cd518f883 Run proc_logs while read_lines_thread in progress 2023-12-20 01:54:25 +08:00
Sean Wei
c7c7df2b58 Process logs in batch 2023-12-20 01:54:25 +08:00
Sean Wei
43ed0c1a53 Implement the parallel version for log parsing 2023-12-20 01:54:25 +08:00
Sean Wei
c612935fa3 Change char * to const char * for parser 2023-12-20 01:54:25 +08:00
Sean Wei
2229709d19 Use logitem instead of glog->item for init_log_item() 
Avoid data race for GLog
 2023-12-20 01:54:25 +08:00
Sean Wei
90be67880d Add GLog->start_time for parse_specifier() and init_log_item() 
Prevent data race for init_log_item()
And reduced the workload of unnecessary localtime_r() function calls
This improvement have x1.15 overall speedup
 2023-12-20 01:54:25 +08:00
Sean Wei
4b140aaf03 Use __sync_add_and_fetch() 
In get_ii32, get_si32, inc_ii32, inc_si32, count_process
 2023-12-20 01:54:25 +08:00
Sean Wei
a8996cc49f Eliminate data races with __sync_bool_compare_and_swap() 
- set conf.* flags in parse_specifier()
- glog->lp.ts = logitem->dt in parse_line()
 2023-12-20 01:54:25 +08:00
Sean Wei
e8ba79e811 Parse lines to GLogItem in batch 
And move count_process() out of parse_line() to avoid data race.
 2023-12-20 01:54:25 +08:00
Sean Wei
49f33081e0 Replace lines with GJob in read_lines() 2023-12-20 01:54:25 +08:00
Sean Wei
18a00656ee Replace logitems with GJob in read_lines() 2023-12-20 01:54:25 +08:00
Sean Wei
22e3f4b11b Add --jobs and --chunk-size options 
And sort getopt short_options list
 2023-12-20 01:54:25 +08:00
Sean Wei
9ee7266d85 Uplift process_log() from read_line() 2023-12-20 01:54:16 +08:00
Sean Wei
6815e30835 Uplift process_log() from parse_line() 2023-12-19 20:09:37 +08:00
Sean Wei
8164121697 Rename pre_process_log() to parse_line() 2023-12-18 13:17:58 +08:00
Sean Wei
df2df1e9d3 Merge read_lines() for with/without GETLINE 2023-12-18 13:17:57 +08:00
Gerardo O
b1332f5a56
Update goaccess.conf 
Ensure we have a valid JSON format
 2023-12-16 12:24:45 -06:00
Gerardo O
cb1613a3aa Fixed TUI and temporarily ignore SIGINT during subdialog execution. 
Closes #2584
 2023-12-13 19:28:30 -06:00
Gerardo O
52bd0940b0
Merge pull request #2591 from Sea-n/master 
Depreacte --output-format, only allow --output option
 2023-12-13 09:53:39 -06:00
Sean Wei
96e64c5da8 Explicitly state that -o support both file and format as argument 2023-12-13 23:41:13 +08:00
Sean Wei
06015717b2 Remove trailing spaces 2023-12-13 23:27:58 +08:00
Sean Wei
b666bfdc7b Depreacte --output-format, only allow --output option 2023-12-13 23:24:19 +08:00
Gerardo O
5138abb0d1
Merge pull request #2592 from Sea-n/fix-gstorage-heap-overflow 
Fix heap overflow in set_browser_os()
 2023-12-11 17:59:03 -06:00
Gerardo O
9561e86b49
Update browsers.c 
Fixed typo.
 2023-12-11 17:21:23 -06:00
Gerardo O
334b053391
Merge pull request #2520 from jordantrizz/new-ua 
enhance(browsers): Updated browsers.c user agents
 2023-12-11 17:20:20 -06:00
Gerardo O
62fe9238dc
Update src/browsers.c 
Co-authored-by: Sean <me@sean.taipei>
 2023-12-11 17:16:59 -06:00
Sean Wei
29c7a6163e Fix heap overflow in set_browser_os() 2023-12-10 19:53:03 +08:00
Gerardo O
0131a7426b
Merge pull request #2588 from Sea-n/master 
Optimize include stmts, Fix IDE warnings and typos
 2023-12-05 13:01:56 -06:00
Sean Wei
b04c6744d5 Slightly improve readme markup 2023-12-06 01:21:48 +08:00
Sean Wei
c47d0f54f6 Use https for gwsocket website link in tpls.html 2023-12-06 01:21:48 +08:00
Sean Wei
c1f3206337 Fix typo in docs 2023-12-06 01:21:48 +08:00
Sean Wei
7a4fb91bb5 Fix typo in source code comments 2023-12-06 01:21:48 +08:00