Commit Graph

358 Commits

Author SHA1 Message Date
Deon George
3dc18a3f1f Change default internal caching to off, and added test to make sure things are cached if on. More debugging with packet name for netmails 2023-01-25 16:26:10 +11:00
Deon George
63e3397aee Add packet name for incoming netmails, dont send back messages to sender for areafix messages 2023-01-24 23:16:11 +11:00
Deon George
bc19f9aa82 Increase logging on the type of file if it isnt a mail packet 2023-01-24 22:03:17 +11:00
Deon George
d9b575b38f Change our determination that a packet has been processed before deciding to delete it 2023-01-24 22:03:17 +11:00
Deon George
dac880b689 Ignore case when checking if a file is a PKT file 2023-01-24 22:03:17 +11:00
Deon George
3392092131 Some comments for binkd 2023-01-11 14:36:40 +11:00
Deon George
724bc4c5af Dont abort on an SQL exception 2023-01-11 14:36:40 +11:00
Deon George
4016775b66 Improved display of bad packets 2023-01-11 14:36:40 +11:00
Deon George
109beb4f4f Enable processing messages coming from a domain with multiple zones 2023-01-02 23:31:31 +11:00
Deon George
c1ab14f2e6 Fix echoarea display with the oldest message 2023-01-01 14:33:52 +11:00
Deon George
d825f71a15 Skip messages that originate from a node in a zone we dont know. 2023-01-01 13:53:51 +11:00
Deon George
06f814395c More debugging to know more about failed echomail parsing because of no zone. 2023-01-01 13:35:56 +11:00
Deon George
6c97eb2d81 Fix for when TZUTC is gibberish, and use file hexdump in pkt debug 2023-01-01 12:02:28 +11:00
Deon George
0e7b93abd1 Fix to make sure we resend the EMSI_REQ banner 2022-12-31 15:13:17 +11:00
Deon George
91fc61e170 Fix nodelist import, when parsing regions 2022-12-19 23:34:42 +11:00
Deon George
eca9f257eb Only show Network Admin if the user is a ZC of a network 2022-12-11 21:46:08 +11:00
Deon George
1656d3c35c Fix address matching during registration 2022-12-11 20:17:24 +11:00
Deon George
f089f3bcdc Dont delete archive until all packets processed 2022-12-04 21:59:06 +11:00
Deon George
4d3d57fdcd Fix EMSI session undefined, other minor cosmetic code changes 2022-12-04 20:45:33 +11:00
Deon George
369f8e48fe Enabled email test command 2022-12-04 14:27:47 +11:00
Deon George
21236b6871 Command to list number of mail waiting for a system 2022-12-04 13:32:46 +11:00
Deon George
05528f1c33 More work on user linking to existing defined system 2022-12-04 13:30:38 +11:00
Deon George
14f28c5263 Fix packet parsing for unknown zones, or zones that are not default 2022-12-03 16:00:38 +11:00
Deon George
a897852072 Fix binkp M_nul OPT parsing 2022-12-03 15:30:31 +11:00
Deon George
09fe65a8db EMSI Continue sending mail until no more mail to send 2022-12-03 01:25:44 +11:00
Deon George
e78b5975b0 Reduce zmodem debugging 2022-12-03 01:25:44 +11:00
Deon George
14349adeab Limit to max messages in a packet, currently hard coded to 50 2022-12-03 01:25:44 +11:00
Deon George
170f5c87ed Log systems polling 2022-12-03 01:25:44 +11:00
Deon George
d5d4a0d781 Auto create Domains/Zones as systems present their AKAs to us 2022-12-03 01:25:44 +11:00
Deon George
55a2a67b8d Auth is now controlled by routing 2022-11-26 06:54:24 +07:00
Deon George
c034ce6cd4 Added FTN list, System View 2022-11-25 17:44:03 +07:00
Deon George
102a972fcb Rename address:merge to address:purge, included merging file/echo subscriptions, updated web subscription cannot add echos to deleted ftns 2022-11-20 17:52:49 +11:00
Deon George
289caa8225 Added address:purge, changing active_addresses index 2022-11-20 17:52:49 +11:00
Deon George
5957a25044 Enabled address undelete and purge, now showing systems deleted messages 2022-11-19 12:02:13 +11:00
Deon George
216bd1a2cd Sometimes guessExtension doesnt return bin for pkt's 2022-11-15 22:13:31 +11:00
Deon George
4e66990826 Fix processing packets when we loop without reading 2022-11-15 22:01:05 +11:00
Deon George
3ffb1c1fd6 Support for ZIP archives 2022-11-14 00:29:55 +11:00
Deon George
702a4e4f37 Attempt to catch DB errors and abort the transaction, so that the queue continues to run. 2022-11-13 15:17:22 +11:00
Deon George
e7ebaa85cb Fix nodelist crc's that have 8 chars 2022-11-13 15:17:22 +11:00
Deon George
b158bb5a7d Add domain to TIC processing, for nodelists that dont have a domain 2022-11-13 15:17:22 +11:00
Deon George
9f0fa242e6 When deleting an Address change active to FALSE 2022-11-13 15:17:22 +11:00
Deon George
eab36c1603 Create TO ftn (for netmails), identify REPLYID, add file:list 2022-11-11 22:57:40 +11:00
Deon George
6284016400 Auto create FTN address for intransit netmail when the destination is not known. 2022-11-06 14:40:03 +11:00
Deon George
9a8ee1aa2b Fix packets with null values in the date, and exceptions not catching it 2022-11-05 19:17:37 +11:00
Deon George
584b988093 UTF8 decoding needs to be done after casting 2022-11-05 15:17:11 +11:00
Deon George
531654724a Enable specifying a domain for nodelist import, and throwing exceptions when errors occur 2022-11-05 15:17:11 +11:00
Deon George
c7187dbb0e Fix casting on compressed attributes when value is NULL, reduce some BINKP debugging messages 2022-11-05 15:17:11 +11:00
Deon George
9721db185b Ensure Echomail is initialised correctly 2022-11-05 10:17:14 +11:00
Deon George
b45e2c6dd8 Support processing nodelists from file repository and assume nodelists are zip fiels 2022-11-05 10:17:14 +11:00
Deon George
2790381a30 Determine if an inbound file is a nodelist 2022-11-05 10:17:14 +11:00
Deon George
34f289953d Implementation fixes for TIC processing 2022-11-05 10:17:14 +11:00
Deon George
7571a2cd7d Enable moving inbound files to s3 and logging in DB 2022-11-04 17:30:18 +11:00
Deon George
029a8a9d73 Added file areas and TIC processing 2022-11-02 22:01:45 +11:00
Deon George
702c5fb4f2 Indexes for performance improvements, improved FTN regex 2022-11-02 22:01:45 +11:00
Deon George
da85e85774 Cast empty Collections to null, Cast strings to zstd compressed strings, add msg_src to echomails processed, fix duplicate seenbys 2022-11-02 21:20:43 +11:00
Deon George
e8fc75f2e3 Remove OldEchomail since we no longer have the mongo dependancies 2022-10-29 17:42:56 +11:00
Deon George
c267573129 Update to Laravel 9 2022-10-27 20:05:51 +08:00
Deon George
e6ba03080b Use zstd for domain home pages info, added nunomaduro/laravel-console-summary 2022-10-27 19:16:35 +08:00
Deon George
5745c67538 Refactor the database, in preparation to moving to postgresql 2022-10-23 13:50:08 +11:00
Deon George
2495e4675c Command refactoring - no functional changes 2022-06-27 00:14:36 +10:00
Deon George
0fe65d6187 Change gethostbyname() with dns_get_record() 2022-03-25 21:50:58 +11:00
Deon George
8072f7c5a9 Work on registration of existing systems to users 2022-03-20 22:42:11 +11:00
Deon George
4ff29e7e43 Some fixes for registering hosts, enable jobs to go to postgres, initial setup fixes 2022-03-12 10:30:02 +11:00
Deon George
cf3b1947a4 Fix creating a system with the correct role, Fix need to have hold/active on creation 2022-03-01 22:07:08 +11:00
Deon George
6db16712ee Fix passthrough netmail. 2022-02-19 17:03:12 +11:00
Deon George
daba4a78d3 Fix echoarea name is converted to UC, add netmail VIA lines for outgoing netmail 2022-02-17 23:55:37 +11:00
Deon George
12a40a79fb Fix for single encoding error, when only the first message was decoded, some logging for Address objects 2022-02-13 11:27:23 +11:00
Deon George
f216f42917 Minor cosmetic changes, show hex_dump when data purged from protocol stream 2022-02-13 11:27:23 +11:00
Deon George
f142284a2b Understand some gated messages, that are gated from a different zone 2022-02-12 11:35:24 +11:00
Deon George
d1bfa3a035 Fix for trying to receive a new file, when the previous file open failed with a permission denied 2022-02-06 20:40:21 +11:00
Deon George
a05ca1c2a4 Misc bug fixes while testing 2022-02-06 20:40:21 +11:00
Deon George
b65289af9f Enable binding ports for IPv6 2022-02-03 13:23:58 +11:00
Deon George
d660d5a6df Move DomainController::NODE* to Address::NODE*, make role mandatory in the database, change logic so that mail generated by the host comes from a node address. 2022-02-03 13:23:58 +11:00
Deon George
efa7195633 Add DBID back to messages, add path/seen-by to generated messages, other minor cosmetic fixes 2022-01-22 23:08:46 +11:00
Deon George
fe9fbb88b0 Fix packet name sent to remote - now matches packet name stored in DB 2022-01-20 23:25:47 +11:00
Deon George
98c9b880d4 Some debugging to track packet names created for sending 2022-01-20 22:47:44 +11:00
Deon George
421cd565bd Enable us to create an outbound packet without updating send details. Determine the send address for packets earlier 2022-01-20 17:54:02 +11:00
Deon George
d930c410dc Remove caching on Zone, fix cockroach cannot do like on integers 2022-01-20 17:51:40 +11:00
Deon George
c7e6bb2575 Enable system mail hold 2022-01-16 11:51:49 +11:00
Deon George
6f1d47a6ab Fixes to message processing, now that we are using cockroachdb 2022-01-15 17:11:51 +11:00
Deon George
e78e79a8f5 Change sorting on graphs so that date is in date order 2022-01-15 14:58:42 +11:00
Deon George
6b0bf32552 Query optimisations for last_messages and traffic 2022-01-15 14:58:42 +11:00
Deon George
3d6f233c1d Added no_export option to stop a message from being exported when saved. 2022-01-15 14:58:42 +11:00
Deon George
847869911f Enable echomail subscription by any of a systems address 2022-01-15 14:58:42 +11:00
Deon George
424d6ef39d Switchout DB to CockroachDB 2022-01-15 14:58:41 +11:00
Deon George
afaa7d8bc7 Internal enhancements to system registration and editing 2022-01-05 09:26:42 +11:00
Deon George
fa2ac9a656 Changes to Authorisation for users to edit systems, minor cosmetic fixes and more logging 2021-12-29 13:44:27 +11:00
Deon George
10afd6f3a4
Limit graphic stats to 6 months by default, some performance query improvements 2021-12-03 11:24:23 +11:00
Deon George
bf57f151d5 Attempt to catch invalid FTN addresses while parsing packets 2021-12-01 23:41:20 +11:00
Deon George
3a1c6d55c6 Enable UTF8 encoding to/from as well in messages 2021-12-01 22:45:51 +11:00
Deon George
f1780e61ea Some messages have the origin in the middle of the message - change strpos to strrpos 2021-11-29 21:12:44 +11:00
Deon George
a00534794e Another fix for when packets contain binary messages with SOH chars. 2021-11-29 00:12:37 +11:00
Deon George
880ff81eff Fix for when packets contain binary messages with SOH chars. Added testing for packet parsing 2021-11-28 22:41:07 +11:00
Deon George
e0239d95a8 Enabled netmail test message 2021-11-27 00:41:09 +11:00
Deon George
b35655a163 Page rendering using ANSI support 2021-11-27 00:40:12 +11:00
Deon George
41d36fe86d
User dashboard caching 2021-11-26 17:19:55 +11:00
Deon George
6acc8ee407
Cache some calls to Mongo for performance 2021-11-26 16:58:50 +11:00
Deon George
be886d9e4b
Add query cache, show per echo stats on network page 2021-11-26 16:16:33 +11:00
Deon George
871430edf8 Replace path from duplicate messages when the existing path is blank 2021-11-25 21:22:36 +11:00
Deon George
ebd1cf8732 Enable duplicates to update seen-by's before being discarded 2021-11-24 22:34:40 +11:00
Deon George
02c23f047c Fix for recording seen-by, add md5 to message body, fix incorrect log of dispatching messages 2021-11-21 16:53:56 +11:00
Deon George
b333d38095 Added in network graph on about screen 2021-11-20 18:00:40 +11:00
Deon George
6f9df005aa Fixes to finding a record when FTN when region_id/host_id have the same value 2021-11-20 11:51:00 +11:00
Deon George
483e35202b Fix for addresses where region_id is set inlieu of host_id, sort system ZC addresses 2021-11-20 11:11:38 +11:00
Deon George
28cdedb327 Centralise dependant js/css files 2021-11-13 17:00:08 +11:00
Deon George
3c8895a238 User register system, minor cosmetic changes, start of user authorisation 2021-11-11 23:04:08 +11:00
Deon George
a0db589dc5 Start on user dashboard 2021-10-26 23:19:55 +11:00
Deon George
8c127ba5da
Try not abort on exception when we cannot dispatch a message to be processed 2021-10-26 17:51:17 +11:00
Deon George
cabf9bca8b Fix for corrupt packets and the object is not properly initialised 2021-10-25 21:09:57 +11:00
Deon George
3adf7c73b6 Change select2 to 4.0.13 and fix autofocus. Other minor cosmetic fix 2021-10-19 22:38:52 +11:00
Deon George
30fd916646 Add user alias 2021-10-19 22:21:32 +11:00
Deon George
53a96a8dbd Change dupe checking as per FTS-0009.001 2021-10-18 23:10:15 +11:00
Deon George
b246f0051f Fix polling, presenting our address. Fix logging address. 2021-10-18 23:08:08 +11:00
Deon George
83e06908b0 Multiple exception bug fixes identified during testing 2021-10-07 23:32:37 +11:00
Deon George
d0d6990d39 Minor cosmetic changes 2021-10-02 10:03:17 +10:00
Deon George
2c406ba3e9 Add $user (authed user) to views 2021-10-02 10:03:17 +10:00
Deon George
ca666e456a Enable user switching 2021-10-02 10:03:17 +10:00
Deon George
fa97fbb0c1 Fix responding to echomail from points 2021-10-02 10:03:17 +10:00
Deon George
eda0926758 Added converting ANSI to a binary format, and custom font rendering 2021-10-02 10:03:17 +10:00
Deon George
82e3283d6d Move address polling to a Job, better catch Socket connection refused errors 2021-09-20 20:39:03 +10:00
Deon George
8b8b513ed1 Add kludge to Test responses 2021-09-14 23:14:13 +10:00
Deon George
2cd6db2060 Dont launch bot processing for rescanned messages 2021-09-13 23:02:39 +10:00
Deon George
79e613a811 Allow for origin/tagline/tearline can have UTF8 chars, show # messages in packets 2021-09-12 23:06:17 +10:00
Deon George
5e8b590f17 Move process functions to Netmail/Echomail folders, optimize Netmail creation when processing messages 2021-09-12 22:14:04 +10:00
Deon George
f91778b515 Fix for when msgid address is from an unknown zone 2021-09-12 22:09:45 +10:00
Deon George
9462e66a1c Some logging fixes 2021-09-12 00:07:02 +10:00
Deon George
fcdde10512 Auto create nodes that are discovered by message packets 2021-09-11 23:32:10 +10:00
Deon George
04e8a899d4 Also ignore nodelist lines that are blank 2021-09-11 19:26:11 +10:00
Deon George
ff2b246d31 For nodelist import, find based on mailer_address first 2021-09-11 18:00:39 +10:00
Deon George
7e176d7bc1 Fix for unknown FTN when connection is not established, try and catch unknown socket exception during session initialisation 2021-09-11 11:47:26 +10:00
Deon George
72bc9790d1 Add option to not process bots during import 2021-09-11 00:40:52 +10:00
Deon George
e8a9508391 Ping/Test send/recevied time was switched 2021-09-10 22:27:00 +10:00
Deon George
fbfa2d5183 Fix Cannot assign null to property when tagline/tearline are blank during export 2021-09-08 22:53:27 +10:00
Deon George
64a887212d Add missed UsePostgres Trait 2021-09-08 22:10:27 +10:00
Deon George
03c4b87cdd Fix packet password on exports, move __unserialize() to EncodeUTF8::decode 2021-09-08 22:07:19 +10:00
Deon George
dbbfe46cb9 Echomail export 2021-09-08 22:07:19 +10:00
Deon George
8306f4c3a3 Start of echomail subscribing 2021-09-06 23:39:48 +10:00
Deon George
97384ce3a0 Fix message path ordering 2021-08-30 00:44:20 +10:00
Deon George
01d54794a8 Rename PacketProcess, fix Packet::open now using Zone not Domain 2021-08-30 00:24:40 +10:00
Deon George
9fb6d191d0 Change the way we figure out zones in packets, some packet testing, fix Echomail import 2021-08-29 23:58:12 +10:00
Deon George
271f066667 Added echomail search and display 2021-08-29 11:48:27 +10:00
Deon George
12f9ee1960 Changed nodelist import processing since we have a DB contraint on mailer address and port 2021-08-28 13:36:42 +10:00
Deon George
6b8704b1cf Added Echoarea NA file import 2021-08-28 00:10:42 +10:00
Deon George
85d9dd1545 Catch Exception when interpretting date, Create MSGID if it doesnt exist, Fix when \r\n is used in messages. 2021-08-27 23:24:31 +10:00
Deon George
ba7f9c1c99 Subject contains UTF-8 codes sometimes 2021-08-26 23:46:49 +10:00
Deon George
403dde0d35 Fix seenby/path addresses, fix when eom signature is split over the readbuf 2021-08-26 22:01:16 +10:00
Deon George
b614779290 Only queue messages if the packet has too many 2021-08-25 22:32:15 +10:00
Deon George
5e3b59ea7c Add subject to encoding, some packet fixes missed in previous commit 2021-08-25 00:15:09 +10:00
Deon George
d86d90cdeb Leverage Redis and queue to handle large packets 2021-08-24 23:42:03 +10:00
Deon George
1398ecff2d
Fix seenaddress accessed before initialisation 2021-08-23 13:35:47 +10:00