Commit Graph

348 Commits

Author SHA1 Message Date
Deon George
d825f71a15 Skip messages that originate from a node in a zone we dont know. 2023-01-01 13:53:51 +11:00
Deon George
06f814395c More debugging to know more about failed echomail parsing because of no zone. 2023-01-01 13:35:56 +11:00
Deon George
6c97eb2d81 Fix for when TZUTC is gibberish, and use file hexdump in pkt debug 2023-01-01 12:02:28 +11:00
Deon George
0e7b93abd1 Fix to make sure we resend the EMSI_REQ banner 2022-12-31 15:13:17 +11:00
Deon George
91fc61e170 Fix nodelist import, when parsing regions 2022-12-19 23:34:42 +11:00
Deon George
eca9f257eb Only show Network Admin if the user is a ZC of a network 2022-12-11 21:46:08 +11:00
Deon George
1656d3c35c Fix address matching during registration 2022-12-11 20:17:24 +11:00
Deon George
f089f3bcdc Dont delete archive until all packets processed 2022-12-04 21:59:06 +11:00
Deon George
4d3d57fdcd Fix EMSI session undefined, other minor cosmetic code changes 2022-12-04 20:45:33 +11:00
Deon George
369f8e48fe Enabled email test command 2022-12-04 14:27:47 +11:00
Deon George
21236b6871 Command to list number of mail waiting for a system 2022-12-04 13:32:46 +11:00
Deon George
05528f1c33 More work on user linking to existing defined system 2022-12-04 13:30:38 +11:00
Deon George
14f28c5263 Fix packet parsing for unknown zones, or zones that are not default 2022-12-03 16:00:38 +11:00
Deon George
a897852072 Fix binkp M_nul OPT parsing 2022-12-03 15:30:31 +11:00
Deon George
09fe65a8db EMSI Continue sending mail until no more mail to send 2022-12-03 01:25:44 +11:00
Deon George
e78b5975b0 Reduce zmodem debugging 2022-12-03 01:25:44 +11:00
Deon George
14349adeab Limit to max messages in a packet, currently hard coded to 50 2022-12-03 01:25:44 +11:00
Deon George
170f5c87ed Log systems polling 2022-12-03 01:25:44 +11:00
Deon George
d5d4a0d781 Auto create Domains/Zones as systems present their AKAs to us 2022-12-03 01:25:44 +11:00
Deon George
55a2a67b8d Auth is now controlled by routing 2022-11-26 06:54:24 +07:00
Deon George
c034ce6cd4 Added FTN list, System View 2022-11-25 17:44:03 +07:00
Deon George
102a972fcb Rename address:merge to address:purge, included merging file/echo subscriptions, updated web subscription cannot add echos to deleted ftns 2022-11-20 17:52:49 +11:00
Deon George
289caa8225 Added address:purge, changing active_addresses index 2022-11-20 17:52:49 +11:00
Deon George
5957a25044 Enabled address undelete and purge, now showing systems deleted messages 2022-11-19 12:02:13 +11:00
Deon George
216bd1a2cd Sometimes guessExtension doesnt return bin for pkt's 2022-11-15 22:13:31 +11:00
Deon George
4e66990826 Fix processing packets when we loop without reading 2022-11-15 22:01:05 +11:00
Deon George
3ffb1c1fd6 Support for ZIP archives 2022-11-14 00:29:55 +11:00
Deon George
702a4e4f37 Attempt to catch DB errors and abort the transaction, so that the queue continues to run. 2022-11-13 15:17:22 +11:00
Deon George
e7ebaa85cb Fix nodelist crc's that have 8 chars 2022-11-13 15:17:22 +11:00
Deon George
b158bb5a7d Add domain to TIC processing, for nodelists that dont have a domain 2022-11-13 15:17:22 +11:00
Deon George
9f0fa242e6 When deleting an Address change active to FALSE 2022-11-13 15:17:22 +11:00
Deon George
eab36c1603 Create TO ftn (for netmails), identify REPLYID, add file:list 2022-11-11 22:57:40 +11:00
Deon George
6284016400 Auto create FTN address for intransit netmail when the destination is not known. 2022-11-06 14:40:03 +11:00
Deon George
9a8ee1aa2b Fix packets with null values in the date, and exceptions not catching it 2022-11-05 19:17:37 +11:00
Deon George
584b988093 UTF8 decoding needs to be done after casting 2022-11-05 15:17:11 +11:00
Deon George
531654724a Enable specifying a domain for nodelist import, and throwing exceptions when errors occur 2022-11-05 15:17:11 +11:00
Deon George
c7187dbb0e Fix casting on compressed attributes when value is NULL, reduce some BINKP debugging messages 2022-11-05 15:17:11 +11:00
Deon George
9721db185b Ensure Echomail is initialised correctly 2022-11-05 10:17:14 +11:00
Deon George
b45e2c6dd8 Support processing nodelists from file repository and assume nodelists are zip fiels 2022-11-05 10:17:14 +11:00
Deon George
2790381a30 Determine if an inbound file is a nodelist 2022-11-05 10:17:14 +11:00
Deon George
34f289953d Implementation fixes for TIC processing 2022-11-05 10:17:14 +11:00
Deon George
7571a2cd7d Enable moving inbound files to s3 and logging in DB 2022-11-04 17:30:18 +11:00
Deon George
029a8a9d73 Added file areas and TIC processing 2022-11-02 22:01:45 +11:00
Deon George
702c5fb4f2 Indexes for performance improvements, improved FTN regex 2022-11-02 22:01:45 +11:00
Deon George
da85e85774 Cast empty Collections to null, Cast strings to zstd compressed strings, add msg_src to echomails processed, fix duplicate seenbys 2022-11-02 21:20:43 +11:00
Deon George
e8fc75f2e3 Remove OldEchomail since we no longer have the mongo dependancies 2022-10-29 17:42:56 +11:00
Deon George
c267573129 Update to Laravel 9 2022-10-27 20:05:51 +08:00
Deon George
e6ba03080b Use zstd for domain home pages info, added nunomaduro/laravel-console-summary 2022-10-27 19:16:35 +08:00
Deon George
5745c67538 Refactor the database, in preparation to moving to postgresql 2022-10-23 13:50:08 +11:00
Deon George
2495e4675c Command refactoring - no functional changes 2022-06-27 00:14:36 +10:00
Deon George
0fe65d6187 Change gethostbyname() with dns_get_record() 2022-03-25 21:50:58 +11:00
Deon George
8072f7c5a9 Work on registration of existing systems to users 2022-03-20 22:42:11 +11:00
Deon George
4ff29e7e43 Some fixes for registering hosts, enable jobs to go to postgres, initial setup fixes 2022-03-12 10:30:02 +11:00
Deon George
cf3b1947a4 Fix creating a system with the correct role, Fix need to have hold/active on creation 2022-03-01 22:07:08 +11:00
Deon George
6db16712ee Fix passthrough netmail. 2022-02-19 17:03:12 +11:00
Deon George
daba4a78d3 Fix echoarea name is converted to UC, add netmail VIA lines for outgoing netmail 2022-02-17 23:55:37 +11:00
Deon George
12a40a79fb Fix for single encoding error, when only the first message was decoded, some logging for Address objects 2022-02-13 11:27:23 +11:00
Deon George
f216f42917 Minor cosmetic changes, show hex_dump when data purged from protocol stream 2022-02-13 11:27:23 +11:00
Deon George
f142284a2b Understand some gated messages, that are gated from a different zone 2022-02-12 11:35:24 +11:00
Deon George
d1bfa3a035 Fix for trying to receive a new file, when the previous file open failed with a permission denied 2022-02-06 20:40:21 +11:00
Deon George
a05ca1c2a4 Misc bug fixes while testing 2022-02-06 20:40:21 +11:00
Deon George
b65289af9f Enable binding ports for IPv6 2022-02-03 13:23:58 +11:00
Deon George
d660d5a6df Move DomainController::NODE* to Address::NODE*, make role mandatory in the database, change logic so that mail generated by the host comes from a node address. 2022-02-03 13:23:58 +11:00
Deon George
efa7195633 Add DBID back to messages, add path/seen-by to generated messages, other minor cosmetic fixes 2022-01-22 23:08:46 +11:00
Deon George
fe9fbb88b0 Fix packet name sent to remote - now matches packet name stored in DB 2022-01-20 23:25:47 +11:00
Deon George
98c9b880d4 Some debugging to track packet names created for sending 2022-01-20 22:47:44 +11:00
Deon George
421cd565bd Enable us to create an outbound packet without updating send details. Determine the send address for packets earlier 2022-01-20 17:54:02 +11:00
Deon George
d930c410dc Remove caching on Zone, fix cockroach cannot do like on integers 2022-01-20 17:51:40 +11:00
Deon George
c7e6bb2575 Enable system mail hold 2022-01-16 11:51:49 +11:00
Deon George
6f1d47a6ab Fixes to message processing, now that we are using cockroachdb 2022-01-15 17:11:51 +11:00
Deon George
e78e79a8f5 Change sorting on graphs so that date is in date order 2022-01-15 14:58:42 +11:00
Deon George
6b0bf32552 Query optimisations for last_messages and traffic 2022-01-15 14:58:42 +11:00
Deon George
3d6f233c1d Added no_export option to stop a message from being exported when saved. 2022-01-15 14:58:42 +11:00
Deon George
847869911f Enable echomail subscription by any of a systems address 2022-01-15 14:58:42 +11:00
Deon George
424d6ef39d Switchout DB to CockroachDB 2022-01-15 14:58:41 +11:00
Deon George
afaa7d8bc7 Internal enhancements to system registration and editing 2022-01-05 09:26:42 +11:00
Deon George
fa2ac9a656 Changes to Authorisation for users to edit systems, minor cosmetic fixes and more logging 2021-12-29 13:44:27 +11:00
Deon George
10afd6f3a4
Limit graphic stats to 6 months by default, some performance query improvements 2021-12-03 11:24:23 +11:00
Deon George
bf57f151d5 Attempt to catch invalid FTN addresses while parsing packets 2021-12-01 23:41:20 +11:00
Deon George
3a1c6d55c6 Enable UTF8 encoding to/from as well in messages 2021-12-01 22:45:51 +11:00
Deon George
f1780e61ea Some messages have the origin in the middle of the message - change strpos to strrpos 2021-11-29 21:12:44 +11:00
Deon George
a00534794e Another fix for when packets contain binary messages with SOH chars. 2021-11-29 00:12:37 +11:00
Deon George
880ff81eff Fix for when packets contain binary messages with SOH chars. Added testing for packet parsing 2021-11-28 22:41:07 +11:00
Deon George
e0239d95a8 Enabled netmail test message 2021-11-27 00:41:09 +11:00
Deon George
b35655a163 Page rendering using ANSI support 2021-11-27 00:40:12 +11:00
Deon George
41d36fe86d
User dashboard caching 2021-11-26 17:19:55 +11:00
Deon George
6acc8ee407
Cache some calls to Mongo for performance 2021-11-26 16:58:50 +11:00
Deon George
be886d9e4b
Add query cache, show per echo stats on network page 2021-11-26 16:16:33 +11:00
Deon George
871430edf8 Replace path from duplicate messages when the existing path is blank 2021-11-25 21:22:36 +11:00
Deon George
ebd1cf8732 Enable duplicates to update seen-by's before being discarded 2021-11-24 22:34:40 +11:00
Deon George
02c23f047c Fix for recording seen-by, add md5 to message body, fix incorrect log of dispatching messages 2021-11-21 16:53:56 +11:00
Deon George
b333d38095 Added in network graph on about screen 2021-11-20 18:00:40 +11:00
Deon George
6f9df005aa Fixes to finding a record when FTN when region_id/host_id have the same value 2021-11-20 11:51:00 +11:00
Deon George
483e35202b Fix for addresses where region_id is set inlieu of host_id, sort system ZC addresses 2021-11-20 11:11:38 +11:00
Deon George
28cdedb327 Centralise dependant js/css files 2021-11-13 17:00:08 +11:00
Deon George
3c8895a238 User register system, minor cosmetic changes, start of user authorisation 2021-11-11 23:04:08 +11:00
Deon George
a0db589dc5 Start on user dashboard 2021-10-26 23:19:55 +11:00
Deon George
8c127ba5da
Try not abort on exception when we cannot dispatch a message to be processed 2021-10-26 17:51:17 +11:00
Deon George
cabf9bca8b Fix for corrupt packets and the object is not properly initialised 2021-10-25 21:09:57 +11:00
Deon George
3adf7c73b6 Change select2 to 4.0.13 and fix autofocus. Other minor cosmetic fix 2021-10-19 22:38:52 +11:00
Deon George
30fd916646 Add user alias 2021-10-19 22:21:32 +11:00
Deon George
53a96a8dbd Change dupe checking as per FTS-0009.001 2021-10-18 23:10:15 +11:00
Deon George
b246f0051f Fix polling, presenting our address. Fix logging address. 2021-10-18 23:08:08 +11:00
Deon George
83e06908b0 Multiple exception bug fixes identified during testing 2021-10-07 23:32:37 +11:00
Deon George
d0d6990d39 Minor cosmetic changes 2021-10-02 10:03:17 +10:00
Deon George
2c406ba3e9 Add $user (authed user) to views 2021-10-02 10:03:17 +10:00
Deon George
ca666e456a Enable user switching 2021-10-02 10:03:17 +10:00
Deon George
fa97fbb0c1 Fix responding to echomail from points 2021-10-02 10:03:17 +10:00
Deon George
eda0926758 Added converting ANSI to a binary format, and custom font rendering 2021-10-02 10:03:17 +10:00
Deon George
82e3283d6d Move address polling to a Job, better catch Socket connection refused errors 2021-09-20 20:39:03 +10:00
Deon George
8b8b513ed1 Add kludge to Test responses 2021-09-14 23:14:13 +10:00
Deon George
2cd6db2060 Dont launch bot processing for rescanned messages 2021-09-13 23:02:39 +10:00
Deon George
79e613a811 Allow for origin/tagline/tearline can have UTF8 chars, show # messages in packets 2021-09-12 23:06:17 +10:00
Deon George
5e8b590f17 Move process functions to Netmail/Echomail folders, optimize Netmail creation when processing messages 2021-09-12 22:14:04 +10:00
Deon George
f91778b515 Fix for when msgid address is from an unknown zone 2021-09-12 22:09:45 +10:00
Deon George
9462e66a1c Some logging fixes 2021-09-12 00:07:02 +10:00
Deon George
fcdde10512 Auto create nodes that are discovered by message packets 2021-09-11 23:32:10 +10:00
Deon George
04e8a899d4 Also ignore nodelist lines that are blank 2021-09-11 19:26:11 +10:00
Deon George
ff2b246d31 For nodelist import, find based on mailer_address first 2021-09-11 18:00:39 +10:00
Deon George
7e176d7bc1 Fix for unknown FTN when connection is not established, try and catch unknown socket exception during session initialisation 2021-09-11 11:47:26 +10:00
Deon George
72bc9790d1 Add option to not process bots during import 2021-09-11 00:40:52 +10:00
Deon George
e8a9508391 Ping/Test send/recevied time was switched 2021-09-10 22:27:00 +10:00
Deon George
fbfa2d5183 Fix Cannot assign null to property when tagline/tearline are blank during export 2021-09-08 22:53:27 +10:00
Deon George
64a887212d Add missed UsePostgres Trait 2021-09-08 22:10:27 +10:00
Deon George
03c4b87cdd Fix packet password on exports, move __unserialize() to EncodeUTF8::decode 2021-09-08 22:07:19 +10:00
Deon George
dbbfe46cb9 Echomail export 2021-09-08 22:07:19 +10:00
Deon George
8306f4c3a3 Start of echomail subscribing 2021-09-06 23:39:48 +10:00
Deon George
97384ce3a0 Fix message path ordering 2021-08-30 00:44:20 +10:00
Deon George
01d54794a8 Rename PacketProcess, fix Packet::open now using Zone not Domain 2021-08-30 00:24:40 +10:00
Deon George
9fb6d191d0 Change the way we figure out zones in packets, some packet testing, fix Echomail import 2021-08-29 23:58:12 +10:00
Deon George
271f066667 Added echomail search and display 2021-08-29 11:48:27 +10:00
Deon George
12f9ee1960 Changed nodelist import processing since we have a DB contraint on mailer address and port 2021-08-28 13:36:42 +10:00
Deon George
6b8704b1cf Added Echoarea NA file import 2021-08-28 00:10:42 +10:00
Deon George
85d9dd1545 Catch Exception when interpretting date, Create MSGID if it doesnt exist, Fix when \r\n is used in messages. 2021-08-27 23:24:31 +10:00
Deon George
ba7f9c1c99 Subject contains UTF-8 codes sometimes 2021-08-26 23:46:49 +10:00
Deon George
403dde0d35 Fix seenby/path addresses, fix when eom signature is split over the readbuf 2021-08-26 22:01:16 +10:00
Deon George
b614779290 Only queue messages if the packet has too many 2021-08-25 22:32:15 +10:00
Deon George
5e3b59ea7c Add subject to encoding, some packet fixes missed in previous commit 2021-08-25 00:15:09 +10:00
Deon George
d86d90cdeb Leverage Redis and queue to handle large packets 2021-08-24 23:42:03 +10:00
Deon George
1398ecff2d
Fix seenaddress accessed before initialisation 2021-08-23 13:35:47 +10:00
Deon George
1ae73e80c2 When MSGID has domain but no point 2021-08-22 16:55:39 +10:00
Deon George
071b817194 Fix processing msgid with @domain 2021-08-22 16:45:09 +10:00
Deon George
e0269c9718 Fix processing packets with echomail messages from points 2021-08-22 16:45:02 +10:00
Deon George
cab402d146 Fix MSGID parsing when it has more than the FTN address 2021-08-22 11:00:41 +10:00
Deon George
8c992ad0f9 Enhance nodelist import 2021-08-22 00:20:22 +10:00
Deon George
68a10af776 Change how our src is determined in packets, add hexdump to packet debug 2021-08-21 00:33:41 +10:00
Deon George
9f762a642c Added SEEN-BY/PATH processing, dont show networks that are not public, minor other formatting 2021-08-19 23:35:48 +10:00
Deon George
ad21285a8c
Tagline/tearline presentation is handled in Message::class 2021-08-19 17:24:56 +10:00
Deon George
8685c6407a
Fix ping blank messages 2021-08-19 16:59:12 +10:00
Deon George
bcf3297fdb Check for duplicate echomails, validate system password 2021-08-19 00:20:34 +10:00