Commit Graph

408 Commits

Author SHA1 Message Date
Deon George
3dc18a3f1f Change default internal caching to off, and added test to make sure things are cached if on. More debugging with packet name for netmails 2023-01-25 16:26:10 +11:00
Deon George
63e3397aee Add packet name for incoming netmails, dont send back messages to sender for areafix messages 2023-01-24 23:16:11 +11:00
Deon George
bc19f9aa82 Increase logging on the type of file if it isnt a mail packet 2023-01-24 22:03:17 +11:00
Deon George
d9b575b38f Change our determination that a packet has been processed before deciding to delete it 2023-01-24 22:03:17 +11:00
Deon George
dac880b689 Ignore case when checking if a file is a PKT file 2023-01-24 22:03:17 +11:00
Deon George
3392092131 Some comments for binkd 2023-01-11 14:36:40 +11:00
Deon George
724bc4c5af Dont abort on an SQL exception 2023-01-11 14:36:40 +11:00
Deon George
4016775b66 Improved display of bad packets 2023-01-11 14:36:40 +11:00
Deon George
109beb4f4f Enable processing messages coming from a domain with multiple zones 2023-01-02 23:31:31 +11:00
Deon George
c1ab14f2e6 Fix echoarea display with the oldest message 2023-01-01 14:33:52 +11:00
Deon George
d825f71a15 Skip messages that originate from a node in a zone we dont know. 2023-01-01 13:53:51 +11:00
Deon George
06f814395c More debugging to know more about failed echomail parsing because of no zone. 2023-01-01 13:35:56 +11:00
Deon George
6c97eb2d81 Fix for when TZUTC is gibberish, and use file hexdump in pkt debug 2023-01-01 12:02:28 +11:00
Deon George
0e7b93abd1 Fix to make sure we resend the EMSI_REQ banner 2022-12-31 15:13:17 +11:00
Deon George
91fc61e170 Fix nodelist import, when parsing regions 2022-12-19 23:34:42 +11:00
Deon George
eca9f257eb Only show Network Admin if the user is a ZC of a network 2022-12-11 21:46:08 +11:00
Deon George
1656d3c35c Fix address matching during registration 2022-12-11 20:17:24 +11:00
Deon George
f089f3bcdc Dont delete archive until all packets processed 2022-12-04 21:59:06 +11:00
Deon George
4d3d57fdcd Fix EMSI session undefined, other minor cosmetic code changes 2022-12-04 20:45:33 +11:00
Deon George
369f8e48fe Enabled email test command 2022-12-04 14:27:47 +11:00
Deon George
21236b6871 Command to list number of mail waiting for a system 2022-12-04 13:32:46 +11:00
Deon George
05528f1c33 More work on user linking to existing defined system 2022-12-04 13:30:38 +11:00
Deon George
14f28c5263 Fix packet parsing for unknown zones, or zones that are not default 2022-12-03 16:00:38 +11:00
Deon George
a897852072 Fix binkp M_nul OPT parsing 2022-12-03 15:30:31 +11:00
Deon George
09fe65a8db EMSI Continue sending mail until no more mail to send 2022-12-03 01:25:44 +11:00
Deon George
e78b5975b0 Reduce zmodem debugging 2022-12-03 01:25:44 +11:00
Deon George
14349adeab Limit to max messages in a packet, currently hard coded to 50 2022-12-03 01:25:44 +11:00
Deon George
170f5c87ed Log systems polling 2022-12-03 01:25:44 +11:00
Deon George
d5d4a0d781 Auto create Domains/Zones as systems present their AKAs to us 2022-12-03 01:25:44 +11:00
Deon George
55a2a67b8d Auth is now controlled by routing 2022-11-26 06:54:24 +07:00
Deon George
c034ce6cd4 Added FTN list, System View 2022-11-25 17:44:03 +07:00
Deon George
102a972fcb Rename address:merge to address:purge, included merging file/echo subscriptions, updated web subscription cannot add echos to deleted ftns 2022-11-20 17:52:49 +11:00
Deon George
289caa8225 Added address:purge, changing active_addresses index 2022-11-20 17:52:49 +11:00
Deon George
5957a25044 Enabled address undelete and purge, now showing systems deleted messages 2022-11-19 12:02:13 +11:00
Deon George
216bd1a2cd Sometimes guessExtension doesnt return bin for pkt's 2022-11-15 22:13:31 +11:00
Deon George
4e66990826 Fix processing packets when we loop without reading 2022-11-15 22:01:05 +11:00
Deon George
3ffb1c1fd6 Support for ZIP archives 2022-11-14 00:29:55 +11:00
Deon George
702a4e4f37 Attempt to catch DB errors and abort the transaction, so that the queue continues to run. 2022-11-13 15:17:22 +11:00
Deon George
e7ebaa85cb Fix nodelist crc's that have 8 chars 2022-11-13 15:17:22 +11:00
Deon George
b158bb5a7d Add domain to TIC processing, for nodelists that dont have a domain 2022-11-13 15:17:22 +11:00
Deon George
9f0fa242e6 When deleting an Address change active to FALSE 2022-11-13 15:17:22 +11:00
Deon George
eab36c1603 Create TO ftn (for netmails), identify REPLYID, add file:list 2022-11-11 22:57:40 +11:00
Deon George
6284016400 Auto create FTN address for intransit netmail when the destination is not known. 2022-11-06 14:40:03 +11:00
Deon George
9a8ee1aa2b Fix packets with null values in the date, and exceptions not catching it 2022-11-05 19:17:37 +11:00
Deon George
584b988093 UTF8 decoding needs to be done after casting 2022-11-05 15:17:11 +11:00
Deon George
531654724a Enable specifying a domain for nodelist import, and throwing exceptions when errors occur 2022-11-05 15:17:11 +11:00
Deon George
c7187dbb0e Fix casting on compressed attributes when value is NULL, reduce some BINKP debugging messages 2022-11-05 15:17:11 +11:00
Deon George
9721db185b Ensure Echomail is initialised correctly 2022-11-05 10:17:14 +11:00
Deon George
b45e2c6dd8 Support processing nodelists from file repository and assume nodelists are zip fiels 2022-11-05 10:17:14 +11:00
Deon George
2790381a30 Determine if an inbound file is a nodelist 2022-11-05 10:17:14 +11:00
Deon George
34f289953d Implementation fixes for TIC processing 2022-11-05 10:17:14 +11:00
Deon George
7571a2cd7d Enable moving inbound files to s3 and logging in DB 2022-11-04 17:30:18 +11:00
Deon George
029a8a9d73 Added file areas and TIC processing 2022-11-02 22:01:45 +11:00
Deon George
702c5fb4f2 Indexes for performance improvements, improved FTN regex 2022-11-02 22:01:45 +11:00
Deon George
da85e85774 Cast empty Collections to null, Cast strings to zstd compressed strings, add msg_src to echomails processed, fix duplicate seenbys 2022-11-02 21:20:43 +11:00
Deon George
e8fc75f2e3 Remove OldEchomail since we no longer have the mongo dependancies 2022-10-29 17:42:56 +11:00
Deon George
c267573129 Update to Laravel 9 2022-10-27 20:05:51 +08:00
Deon George
e6ba03080b Use zstd for domain home pages info, added nunomaduro/laravel-console-summary 2022-10-27 19:16:35 +08:00
Deon George
5745c67538 Refactor the database, in preparation to moving to postgresql 2022-10-23 13:50:08 +11:00
Deon George
2495e4675c Command refactoring - no functional changes 2022-06-27 00:14:36 +10:00
Deon George
0fe65d6187 Change gethostbyname() with dns_get_record() 2022-03-25 21:50:58 +11:00
Deon George
8072f7c5a9 Work on registration of existing systems to users 2022-03-20 22:42:11 +11:00
Deon George
4ff29e7e43 Some fixes for registering hosts, enable jobs to go to postgres, initial setup fixes 2022-03-12 10:30:02 +11:00
Deon George
cf3b1947a4 Fix creating a system with the correct role, Fix need to have hold/active on creation 2022-03-01 22:07:08 +11:00
Deon George
6db16712ee Fix passthrough netmail. 2022-02-19 17:03:12 +11:00
Deon George
daba4a78d3 Fix echoarea name is converted to UC, add netmail VIA lines for outgoing netmail 2022-02-17 23:55:37 +11:00
Deon George
12a40a79fb Fix for single encoding error, when only the first message was decoded, some logging for Address objects 2022-02-13 11:27:23 +11:00
Deon George
f216f42917 Minor cosmetic changes, show hex_dump when data purged from protocol stream 2022-02-13 11:27:23 +11:00
Deon George
f142284a2b Understand some gated messages, that are gated from a different zone 2022-02-12 11:35:24 +11:00
Deon George
d1bfa3a035 Fix for trying to receive a new file, when the previous file open failed with a permission denied 2022-02-06 20:40:21 +11:00
Deon George
a05ca1c2a4 Misc bug fixes while testing 2022-02-06 20:40:21 +11:00
Deon George
b65289af9f Enable binding ports for IPv6 2022-02-03 13:23:58 +11:00
Deon George
d660d5a6df Move DomainController::NODE* to Address::NODE*, make role mandatory in the database, change logic so that mail generated by the host comes from a node address. 2022-02-03 13:23:58 +11:00
Deon George
efa7195633 Add DBID back to messages, add path/seen-by to generated messages, other minor cosmetic fixes 2022-01-22 23:08:46 +11:00
Deon George
fe9fbb88b0 Fix packet name sent to remote - now matches packet name stored in DB 2022-01-20 23:25:47 +11:00
Deon George
98c9b880d4 Some debugging to track packet names created for sending 2022-01-20 22:47:44 +11:00
Deon George
421cd565bd Enable us to create an outbound packet without updating send details. Determine the send address for packets earlier 2022-01-20 17:54:02 +11:00
Deon George
d930c410dc Remove caching on Zone, fix cockroach cannot do like on integers 2022-01-20 17:51:40 +11:00
Deon George
c7e6bb2575 Enable system mail hold 2022-01-16 11:51:49 +11:00
Deon George
6f1d47a6ab Fixes to message processing, now that we are using cockroachdb 2022-01-15 17:11:51 +11:00
Deon George
e78e79a8f5 Change sorting on graphs so that date is in date order 2022-01-15 14:58:42 +11:00
Deon George
6b0bf32552 Query optimisations for last_messages and traffic 2022-01-15 14:58:42 +11:00
Deon George
3d6f233c1d Added no_export option to stop a message from being exported when saved. 2022-01-15 14:58:42 +11:00
Deon George
847869911f Enable echomail subscription by any of a systems address 2022-01-15 14:58:42 +11:00
Deon George
424d6ef39d Switchout DB to CockroachDB 2022-01-15 14:58:41 +11:00
Deon George
afaa7d8bc7 Internal enhancements to system registration and editing 2022-01-05 09:26:42 +11:00
Deon George
fa2ac9a656 Changes to Authorisation for users to edit systems, minor cosmetic fixes and more logging 2021-12-29 13:44:27 +11:00
Deon George
10afd6f3a4
Limit graphic stats to 6 months by default, some performance query improvements 2021-12-03 11:24:23 +11:00
Deon George
bf57f151d5 Attempt to catch invalid FTN addresses while parsing packets 2021-12-01 23:41:20 +11:00
Deon George
3a1c6d55c6 Enable UTF8 encoding to/from as well in messages 2021-12-01 22:45:51 +11:00
Deon George
f1780e61ea Some messages have the origin in the middle of the message - change strpos to strrpos 2021-11-29 21:12:44 +11:00
Deon George
a00534794e Another fix for when packets contain binary messages with SOH chars. 2021-11-29 00:12:37 +11:00
Deon George
880ff81eff Fix for when packets contain binary messages with SOH chars. Added testing for packet parsing 2021-11-28 22:41:07 +11:00
Deon George
e0239d95a8 Enabled netmail test message 2021-11-27 00:41:09 +11:00
Deon George
b35655a163 Page rendering using ANSI support 2021-11-27 00:40:12 +11:00
Deon George
41d36fe86d
User dashboard caching 2021-11-26 17:19:55 +11:00
Deon George
6acc8ee407
Cache some calls to Mongo for performance 2021-11-26 16:58:50 +11:00
Deon George
be886d9e4b
Add query cache, show per echo stats on network page 2021-11-26 16:16:33 +11:00
Deon George
871430edf8 Replace path from duplicate messages when the existing path is blank 2021-11-25 21:22:36 +11:00
Deon George
ebd1cf8732 Enable duplicates to update seen-by's before being discarded 2021-11-24 22:34:40 +11:00
Deon George
02c23f047c Fix for recording seen-by, add md5 to message body, fix incorrect log of dispatching messages 2021-11-21 16:53:56 +11:00
Deon George
b333d38095 Added in network graph on about screen 2021-11-20 18:00:40 +11:00
Deon George
6f9df005aa Fixes to finding a record when FTN when region_id/host_id have the same value 2021-11-20 11:51:00 +11:00
Deon George
483e35202b Fix for addresses where region_id is set inlieu of host_id, sort system ZC addresses 2021-11-20 11:11:38 +11:00
Deon George
28cdedb327 Centralise dependant js/css files 2021-11-13 17:00:08 +11:00
Deon George
3c8895a238 User register system, minor cosmetic changes, start of user authorisation 2021-11-11 23:04:08 +11:00
Deon George
a0db589dc5 Start on user dashboard 2021-10-26 23:19:55 +11:00
Deon George
8c127ba5da
Try not abort on exception when we cannot dispatch a message to be processed 2021-10-26 17:51:17 +11:00
Deon George
cabf9bca8b Fix for corrupt packets and the object is not properly initialised 2021-10-25 21:09:57 +11:00
Deon George
3adf7c73b6 Change select2 to 4.0.13 and fix autofocus. Other minor cosmetic fix 2021-10-19 22:38:52 +11:00
Deon George
30fd916646 Add user alias 2021-10-19 22:21:32 +11:00
Deon George
53a96a8dbd Change dupe checking as per FTS-0009.001 2021-10-18 23:10:15 +11:00
Deon George
b246f0051f Fix polling, presenting our address. Fix logging address. 2021-10-18 23:08:08 +11:00
Deon George
83e06908b0 Multiple exception bug fixes identified during testing 2021-10-07 23:32:37 +11:00
Deon George
d0d6990d39 Minor cosmetic changes 2021-10-02 10:03:17 +10:00
Deon George
2c406ba3e9 Add $user (authed user) to views 2021-10-02 10:03:17 +10:00
Deon George
ca666e456a Enable user switching 2021-10-02 10:03:17 +10:00
Deon George
fa97fbb0c1 Fix responding to echomail from points 2021-10-02 10:03:17 +10:00
Deon George
eda0926758 Added converting ANSI to a binary format, and custom font rendering 2021-10-02 10:03:17 +10:00
Deon George
82e3283d6d Move address polling to a Job, better catch Socket connection refused errors 2021-09-20 20:39:03 +10:00
Deon George
8b8b513ed1 Add kludge to Test responses 2021-09-14 23:14:13 +10:00
Deon George
2cd6db2060 Dont launch bot processing for rescanned messages 2021-09-13 23:02:39 +10:00
Deon George
79e613a811 Allow for origin/tagline/tearline can have UTF8 chars, show # messages in packets 2021-09-12 23:06:17 +10:00
Deon George
5e8b590f17 Move process functions to Netmail/Echomail folders, optimize Netmail creation when processing messages 2021-09-12 22:14:04 +10:00
Deon George
f91778b515 Fix for when msgid address is from an unknown zone 2021-09-12 22:09:45 +10:00
Deon George
9462e66a1c Some logging fixes 2021-09-12 00:07:02 +10:00
Deon George
fcdde10512 Auto create nodes that are discovered by message packets 2021-09-11 23:32:10 +10:00
Deon George
04e8a899d4 Also ignore nodelist lines that are blank 2021-09-11 19:26:11 +10:00
Deon George
ff2b246d31 For nodelist import, find based on mailer_address first 2021-09-11 18:00:39 +10:00
Deon George
7e176d7bc1 Fix for unknown FTN when connection is not established, try and catch unknown socket exception during session initialisation 2021-09-11 11:47:26 +10:00
Deon George
72bc9790d1 Add option to not process bots during import 2021-09-11 00:40:52 +10:00
Deon George
e8a9508391 Ping/Test send/recevied time was switched 2021-09-10 22:27:00 +10:00
Deon George
fbfa2d5183 Fix Cannot assign null to property when tagline/tearline are blank during export 2021-09-08 22:53:27 +10:00
Deon George
64a887212d Add missed UsePostgres Trait 2021-09-08 22:10:27 +10:00
Deon George
03c4b87cdd Fix packet password on exports, move __unserialize() to EncodeUTF8::decode 2021-09-08 22:07:19 +10:00
Deon George
dbbfe46cb9 Echomail export 2021-09-08 22:07:19 +10:00
Deon George
8306f4c3a3 Start of echomail subscribing 2021-09-06 23:39:48 +10:00
Deon George
97384ce3a0 Fix message path ordering 2021-08-30 00:44:20 +10:00
Deon George
01d54794a8 Rename PacketProcess, fix Packet::open now using Zone not Domain 2021-08-30 00:24:40 +10:00
Deon George
9fb6d191d0 Change the way we figure out zones in packets, some packet testing, fix Echomail import 2021-08-29 23:58:12 +10:00
Deon George
271f066667 Added echomail search and display 2021-08-29 11:48:27 +10:00
Deon George
12f9ee1960 Changed nodelist import processing since we have a DB contraint on mailer address and port 2021-08-28 13:36:42 +10:00
Deon George
6b8704b1cf Added Echoarea NA file import 2021-08-28 00:10:42 +10:00
Deon George
85d9dd1545 Catch Exception when interpretting date, Create MSGID if it doesnt exist, Fix when \r\n is used in messages. 2021-08-27 23:24:31 +10:00
Deon George
ba7f9c1c99 Subject contains UTF-8 codes sometimes 2021-08-26 23:46:49 +10:00
Deon George
403dde0d35 Fix seenby/path addresses, fix when eom signature is split over the readbuf 2021-08-26 22:01:16 +10:00
Deon George
b614779290 Only queue messages if the packet has too many 2021-08-25 22:32:15 +10:00
Deon George
5e3b59ea7c Add subject to encoding, some packet fixes missed in previous commit 2021-08-25 00:15:09 +10:00
Deon George
d86d90cdeb Leverage Redis and queue to handle large packets 2021-08-24 23:42:03 +10:00
Deon George
1398ecff2d
Fix seenaddress accessed before initialisation 2021-08-23 13:35:47 +10:00
Deon George
1ae73e80c2 When MSGID has domain but no point 2021-08-22 16:55:39 +10:00
Deon George
071b817194 Fix processing msgid with @domain 2021-08-22 16:45:09 +10:00
Deon George
e0269c9718 Fix processing packets with echomail messages from points 2021-08-22 16:45:02 +10:00
Deon George
cab402d146 Fix MSGID parsing when it has more than the FTN address 2021-08-22 11:00:41 +10:00
Deon George
8c992ad0f9 Enhance nodelist import 2021-08-22 00:20:22 +10:00
Deon George
68a10af776 Change how our src is determined in packets, add hexdump to packet debug 2021-08-21 00:33:41 +10:00
Deon George
9f762a642c Added SEEN-BY/PATH processing, dont show networks that are not public, minor other formatting 2021-08-19 23:35:48 +10:00
Deon George
ad21285a8c
Tagline/tearline presentation is handled in Message::class 2021-08-19 17:24:56 +10:00
Deon George
8685c6407a
Fix ping blank messages 2021-08-19 16:59:12 +10:00
Deon George
bcf3297fdb Check for duplicate echomails, validate system password 2021-08-19 00:20:34 +10:00
Deon George
6af466605a Logging updates 2021-08-18 22:11:16 +10:00
Deon George
259ea01fd1 Layout changes by responses, fix NodeHeirarchy domains nolonger have default 2021-08-18 00:03:44 +10:00
Deon George
978843b5e3 Simplified logging and fix BINKP issue when receiving less than the blksize 2021-08-17 23:49:39 +10:00
Deon George
5bf612e5b4 Set packed boolean when packing Netmail's is successful 2021-08-16 22:30:34 +10:00
Deon George
628293c741 Enable default zones for 4d systems 2021-08-16 22:26:33 +10:00
Deon George
111461e515 Logging updates, some function optimisation 2021-08-16 00:41:43 +10:00
Deon George
b8478adecb Catch Netmails that dont generate an exception when converting to a packed message. Make sure we present unique addresses 2021-08-15 19:47:33 +10:00
Deon George
0789ee9042 Fix assigning default when system has more than 1 AKA in different zones 2021-08-15 17:26:01 +10:00
Deon George
988d4dda52 Sort node's AKAs, fix Host selection for new addresses 2021-08-15 16:41:04 +10:00
Deon George
ae96ff9a24 Fix System ZC identification 2021-08-15 16:01:51 +10:00
Deon George
414a10f84e Fix end session report, move address when system is a ZC, show #addresses on systems 2021-08-15 11:42:38 +10:00
Deon George
b9abd17139 Dont process messages from robots 2021-08-15 00:21:04 +10:00
Deon George
3badb895b0 Filearea validation fixes 2021-08-14 23:52:39 +10:00
Deon George
c904f1f9e9 Fix echoarea naming regex 2021-08-14 23:05:11 +10:00
Deon George
4048289cf3 Implemented hide AKA, Minor Node::class changes, other fixes 2021-08-14 16:14:43 +10:00
Deon George
569201e972 Server start based on config 2021-08-14 16:14:43 +10:00
Deon George
6c6976678f Change joins with a collection to use collection joins 2021-08-14 11:23:04 +10:00
Deon George
385290d18c Handle packets with errors differently 2021-08-13 23:46:48 +10:00
Deon George
05f387aab7 Assigning session details to ZC makes it default route 2021-08-13 22:53:59 +10:00
Deon George
53ab8851fa Add packet info console command 2021-08-12 23:15:45 +10:00
Deon George
393600f96f Dont process incomplete files 2021-08-12 21:59:48 +10:00
Deon George
eb0405f019 Added echoareas and fileareas 2021-08-11 23:45:49 +10:00
Deon George
c7388c2db6 Enabled default routing 2021-08-09 23:35:22 +10:00
Deon George
7ec01d778a System::match() now returns an array 2021-08-08 23:53:56 +10:00
Deon George
7922bc6a7a Fix errors must not be accessed before initialization 2021-08-08 22:49:38 +10:00
Deon George
72fb8044f9 Fix for identifying children for netmail packets 2021-08-08 21:50:09 +10:00
Deon George
8d4ca2963d Enabled moving address, minor CSS fixes 2021-08-08 17:27:35 +10:00
Deon George
7cd7849241 Move packet debug view to top level 2021-08-08 12:01:10 +10:00
Deon George
b179b1b3e9 Mail routing parent/children, domain name validation, nodelist import changes and other fixes 2021-08-08 01:14:11 +10:00
Deon George
49bc946024 CSS changes, enable links to system edit on network list 2021-08-08 01:12:11 +10:00
Deon George
d937547599 Some message optimisation, added Echomail processing 2021-07-31 00:44:23 +10:00
Deon George
6f26bc4c71 Validation fix when creating a new host 2021-07-31 00:43:25 +10:00
Deon George
29a707831e CSS fixes, BS5 doesnt have ml/mr,pl/pr. Fix remember me. Style consistencies. Added Error 500 page. 2021-07-24 14:12:10 +10:00
Deon George
bbbb06d0a2 Framework update and minor packet debug cosmetics 2021-07-24 00:54:39 +10:00
Deon George
ee30ef92c3 New algorithm for calculating packet name, EMSI/BINKP inbound processing tested, Netmail rejection and intransit processing 2021-07-24 00:53:35 +10:00
Deon George
2fdc6eabad Revise CRC calls and use php's internal functions 2021-07-21 20:52:17 +10:00
Deon George
7cd3b814bb Remove processed packets if there were no issues 2021-07-20 00:26:12 +10:00
Deon George
6027ce52ab Fix child processes exiting 2021-07-20 00:23:41 +10:00
Deon George
ccab973b4a Add missing ls_recvdata16 to ZMODEM 2021-07-19 22:15:09 +10:00
Deon George
a37da4b099 Validation fix when creating a new host 2021-07-18 23:38:18 +10:00