pingpong.git
3 years agoExtract ClusterMatcherObserver interface to separate file and move code that handles...
Janus Varmarken [Mon, 14 Jan 2019 23:54:36 +0000 (15:54 -0800)]
Extract ClusterMatcherObserver interface to separate file and move code that handles observer (de-)registration to AbstractClusterMatcher.

3 years agoLayer2SequenceMatcher: check packet directions when matching sequence.
Janus Varmarken [Sun, 13 Jan 2019 22:16:44 +0000 (14:16 -0800)]
Layer2SequenceMatcher: check packet directions when matching sequence.

3 years agoMake original ClusterMatcher inherit from AbstractClusterMatcher. Reorganize code...
Janus Varmarken [Sun, 13 Jan 2019 05:03:30 +0000 (21:03 -0800)]
Make original ClusterMatcher inherit from AbstractClusterMatcher. Reorganize code by adding subpackages for layer2 and layer3 detection.

3 years agoReorganize code by creating a package for code that reassembles traffic flows at...
Janus Varmarken [Sun, 13 Jan 2019 04:18:53 +0000 (20:18 -0800)]
Reorganize code by creating a package for code that reassembles traffic flows at different layers

3 years agoCleanup of layer 2 sequence detection: get rid of some experimental classes
Janus Varmarken [Sun, 13 Jan 2019 04:05:29 +0000 (20:05 -0800)]
Cleanup of layer 2 sequence detection: get rid of some experimental classes

3 years agoFirst work on layer 2 sequence matching. Basic functionality seems to work. Cleanup...
Janus Varmarken [Sun, 13 Jan 2019 03:58:13 +0000 (19:58 -0800)]
First work on layer 2 sequence matching. Basic functionality seems to work. Cleanup needed.

3 years agoChecking in new files and new entries for Blossom sprinkler.
rtrimana [Fri, 11 Jan 2019 02:08:04 +0000 (18:08 -0800)]
Checking in new files and new entries for Blossom sprinkler.

3 years agoClearing up conflicts.
rtrimana [Wed, 28 Nov 2018 16:16:57 +0000 (08:16 -0800)]
Clearing up conflicts.

3 years agoMerging changes.
rtrimana [Wed, 28 Nov 2018 16:14:54 +0000 (08:14 -0800)]
Merging changes.

3 years agoSignatureDetector: get filenames from program arguments in order to make execution...
Janus Varmarken [Wed, 28 Nov 2018 15:53:02 +0000 (07:53 -0800)]
SignatureDetector: get filenames from program arguments in order to make execution on Zeus simpler (avoid push + pull whenever we explore a new signature on a new file)

3 years agoadded a hacky solution for extracting a sanity-check signature to be run against...
Janus Varmarken [Wed, 28 Nov 2018 07:05:25 +0000 (23:05 -0800)]
added a hacky solution for extracting a sanity-check signature to be run against negative test sets

3 years agoIMPORTANT BUGFIX: consider FULL range of local IPs when determining clietnIsSrc in...
Janus Varmarken [Wed, 28 Nov 2018 06:57:26 +0000 (22:57 -0800)]
IMPORTANT BUGFIX: consider FULL range of local IPs when determining clietnIsSrc in TcpReassembler

3 years agoPoint Zeus to ST plug signatures
Janus Varmarken [Wed, 28 Nov 2018 04:49:24 +0000 (20:49 -0800)]
Point Zeus to ST plug signatures

3 years agoAdd output that displays what files are in use; point Zeus to kwikset signatures.
Janus Varmarken [Wed, 28 Nov 2018 04:11:50 +0000 (20:11 -0800)]
Add output that displays what files are in use; point Zeus to kwikset signatures.

3 years agoPrepare code for execution on Zeus (including upping mem limit to 300gigs)
Janus Varmarken [Wed, 28 Nov 2018 03:14:22 +0000 (19:14 -0800)]
Prepare code for execution on Zeus (including upping mem limit to 300gigs)

3 years agoRevert back to full MondayWorkhours file; increase Xmx to 64GB as default when run...
Janus Varmarken [Wed, 28 Nov 2018 01:16:10 +0000 (17:16 -0800)]
Revert back to full MondayWorkhours file; increase Xmx to 64GB as default when run using gradle run

3 years agoPrepare code for execution on Hera (networking group server)
Janus Varmarken [Tue, 27 Nov 2018 21:35:17 +0000 (13:35 -0800)]
Prepare code for execution on Hera (networking group server)

3 years agoAdding WeMo Inisght paths for experimental setup.
rtrimana [Fri, 23 Nov 2018 19:58:26 +0000 (11:58 -0800)]
Adding WeMo Inisght paths for experimental setup.

3 years agoAdding WeMo plugs' paths for experimental results.
rtrimana [Thu, 22 Nov 2018 16:17:54 +0000 (08:17 -0800)]
Adding WeMo plugs' paths for experimental results.

3 years agoAdding new smart home devices into the experiment.
rtrimana [Wed, 21 Nov 2018 21:13:49 +0000 (13:13 -0800)]
Adding new smart home devices into the experiment.

3 years agoAdding new devices under test.
rtrimana [Sat, 17 Nov 2018 01:09:53 +0000 (17:09 -0800)]
Adding new devices under test.

3 years agoAdding more devices into the experimental setup.
rtrimana [Tue, 13 Nov 2018 23:38:25 +0000 (15:38 -0800)]
Adding more devices into the experimental setup.

3 years agoAdding more execution parameters for experimental results (evaluation).
rtrimana [Sun, 11 Nov 2018 03:45:24 +0000 (19:45 -0800)]
Adding more execution parameters for experimental results (evaluation).

3 years agoprune clusters in order to allow detection of different pairs in same connection
Janus Varmarken [Thu, 8 Nov 2018 18:34:52 +0000 (10:34 -0800)]
prune clusters in order to allow detection of different pairs in same connection

3 years agoAdding a method to delete a bad sequence in a signature after we test the produced...
rtrimana [Wed, 7 Nov 2018 19:03:20 +0000 (11:03 -0800)]
Adding a method to delete a bad sequence in a signature after we test the produced signature file against its own PCAP file---one case like this is the Nest Thermostat.

3 years agoFixing curl for twitter.com in browser.sh.
rtrimana [Wed, 7 Nov 2018 00:52:31 +0000 (16:52 -0800)]
Fixing curl for twitter.com in browser.sh.

3 years agoAdding more websites to the fake browser.
rtrimana [Wed, 7 Nov 2018 00:49:12 +0000 (16:49 -0800)]
Adding more websites to the fake browser.

3 years agoAdding a dummy browser script.
rtrimana [Mon, 5 Nov 2018 19:35:13 +0000 (11:35 -0800)]
Adding a dummy browser script.

3 years agoTemporary fix for router's IP as client for WAN trace and paths for new tests for...
rtrimana [Fri, 2 Nov 2018 23:05:14 +0000 (16:05 -0700)]
Temporary fix for router's IP as client for WAN trace and paths for new tests for devices.

3 years agoMaking sure that merging would fail if there is a situation where two sequences are...
rtrimana [Wed, 31 Oct 2018 23:28:21 +0000 (16:28 -0700)]
Making sure that merging would fail if there is a situation where two sequences are sometimes adjacent and sometimes aren't. The detection part will detect these, instead, as separate ordered sequences (that are in one TCP connection).

3 years agoChecking in new code for signature generation; Arlo has a signature for camera on...
rtrimana [Wed, 31 Oct 2018 16:56:59 +0000 (09:56 -0700)]
Checking in new code for signature generation; Arlo has a signature for camera on/off! :)

3 years agoAdding a temporary cleanup for duplicate timestamps due to ON and OFF signatures...
rtrimana [Thu, 18 Oct 2018 22:07:06 +0000 (15:07 -0700)]
Adding a temporary cleanup for duplicate timestamps due to ON and OFF signatures having the same sequences (and thus the same event is detected and printed twice).

3 years agoAdding array out of bound check for timestamps comparison.
rtrimana [Thu, 18 Oct 2018 20:20:30 +0000 (13:20 -0700)]
Adding array out of bound check for timestamps comparison.

3 years agoTesting the flow starting from D-Link plug for training and signature generation.
rtrimana [Thu, 18 Oct 2018 20:19:38 +0000 (13:19 -0700)]
Testing the flow starting from D-Link plug for training and signature generation.

3 years agoAdding files for training for paper evaluation.
rtrimana [Wed, 17 Oct 2018 23:45:32 +0000 (16:45 -0700)]
Adding files for training for paper evaluation.

3 years agoAdding PCAP file for no-activity.
rtrimana [Thu, 11 Oct 2018 00:01:19 +0000 (17:01 -0700)]
Adding PCAP file for no-activity.

3 years agoAdding a script to compare 2 timestamps files to validate the detection process.
rtrimana [Tue, 9 Oct 2018 23:39:42 +0000 (16:39 -0700)]
Adding a script to compare 2 timestamps files to validate the detection process.

3 years agoSignatureDetector: add paths to dlink plug evaluation experiment
Janus Varmarken [Tue, 9 Oct 2018 17:32:12 +0000 (10:32 -0700)]
SignatureDetector: add paths to dlink plug evaluation experiment

3 years agoTcpReassembler: skip non-IPv4 traffic; SignatureDetector: add paths to d-link evaluat...
Janus Varmarken [Tue, 9 Oct 2018 17:31:13 +0000 (10:31 -0700)]
TcpReassembler: skip non-IPv4 traffic; SignatureDetector: add paths to d-link evaluation experiment

3 years agoFixing conflict for SignatureDetector.java.
rtrimana [Tue, 9 Oct 2018 00:42:33 +0000 (17:42 -0700)]
Fixing conflict for SignatureDetector.java.

3 years agoCommitting local changes.
rtrimana [Tue, 9 Oct 2018 00:03:51 +0000 (17:03 -0700)]
Committing local changes.

3 years agoadd filepaths for dlink siren and dlink plug to SignatureDetector's embedded test...
Janus Varmarken [Wed, 3 Oct 2018 22:17:13 +0000 (15:17 -0700)]
add filepaths for dlink siren and dlink plug to SignatureDetector's embedded test client

3 years agoSignatureDetector.java cleanup: remove initial attempts at algorithm; add some javadoc.
Janus Varmarken [Tue, 2 Oct 2018 23:39:21 +0000 (16:39 -0700)]
SignatureDetector.java cleanup: remove initial attempts at algorithm; add some javadoc.

3 years agoAdded SignatureDetector which can detect signatures spanning multiple TCP connections
Janus Varmarken [Tue, 2 Oct 2018 23:32:27 +0000 (16:32 -0700)]
Added SignatureDetector which can detect signatures spanning multiple TCP connections

3 years agoClusterMatcher.java: cleanup; add javadoc; change signature of observer interface
Janus Varmarken [Wed, 26 Sep 2018 22:47:51 +0000 (15:47 -0700)]
ClusterMatcher.java: cleanup; add javadoc; change signature of observer interface

3 years agorename SignatureDetector to ClusterMatcher
Janus Varmarken [Wed, 26 Sep 2018 19:54:32 +0000 (12:54 -0700)]
rename SignatureDetector to ClusterMatcher

3 years agoAdding the serializer/deserializer for the entire signature.
rtrimana [Wed, 26 Sep 2018 18:58:03 +0000 (11:58 -0700)]
Adding the serializer/deserializer for the entire signature.

3 years agoSignatureDetector.java: now also detects when the same sequence occurs multiple times...
Janus Varmarken [Wed, 26 Sep 2018 01:28:22 +0000 (18:28 -0700)]
SignatureDetector.java: now also detects when the same sequence occurs multiple times in the same conversation.

3 years agobacking up first "working" version of SignatureDetector; note that this version does...
Janus Varmarken [Tue, 25 Sep 2018 23:49:34 +0000 (16:49 -0700)]
backing up first "working" version of SignatureDetector; note that this version does not manage to capture those cases where the signature appears multiple times in the same TCP Conversation.

3 years agoSignatureDetector.java: closer towards final implementation (e.g., also check packet...
Janus Varmarken [Sat, 22 Sep 2018 00:29:59 +0000 (17:29 -0700)]
SignatureDetector.java: closer towards final implementation (e.g., also check packet direction when detecting signatures)

3 years agoPcapPacketUtils.java: change implementation of getSourceIp(PcapPacket) to throw a...
Janus Varmarken [Fri, 21 Sep 2018 23:00:48 +0000 (16:00 -0700)]
PcapPacketUtils.java: change implementation of getSourceIp(PcapPacket) to throw a NullPointerException instead of returning null if the provided packet does not contain an IpV4Packet; add implementations of getDestinationIp(PcapPacket), isSrcIpLocal(PcapPacket), isDstIpLocal(PcapPacket), getIpV4PacketOrThrow(PcapPacket).

3 years agoAdding signature sorting by timestamps.
rtrimana [Sat, 22 Sep 2018 00:00:12 +0000 (17:00 -0700)]
Adding signature sorting by timestamps.

3 years agoPerformance bug found---unnecessary outer loop introduced.
rtrimana [Fri, 21 Sep 2018 21:01:12 +0000 (14:01 -0700)]
Performance bug found---unnecessary outer loop introduced.

3 years ago[minor work in progress]
Janus Varmarken [Fri, 21 Sep 2018 17:56:27 +0000 (10:56 -0700)]
[minor work in progress]

3 years agopreliminary work on signature detection
Janus Varmarken [Thu, 20 Sep 2018 23:45:39 +0000 (16:45 -0700)]
preliminary work on signature detection

3 years agoAdding signature combining; we can improve the performance later by requiring that...
rtrimana [Thu, 20 Sep 2018 23:29:08 +0000 (16:29 -0700)]
Adding signature combining; we can improve the performance later by requiring that the Conversation list should always be sorted based on timestamps.

3 years agoSupport for starting a live capture.
Janus Varmarken [Wed, 19 Sep 2018 22:11:10 +0000 (15:11 -0700)]
Support for starting a live capture.
Support for terminating PcapHandleReader.

3 years agoOnly taking signatures in the range of trigger+/-5; pad with null if the second eleme...
rtrimana [Wed, 19 Sep 2018 18:12:04 +0000 (11:12 -0700)]
Only taking signatures in the range of trigger+/-5; pad with null if the second element of the pair is null.

3 years agoAdding the basic signature harvesting (i.e., also saving to and reading back from...
rtrimana [Wed, 19 Sep 2018 16:54:30 +0000 (09:54 -0700)]
Adding the basic signature harvesting (i.e., also saving to and reading back from signature files).

3 years agoResolving conflict.
rtrimana [Mon, 17 Sep 2018 23:30:54 +0000 (16:30 -0700)]
Resolving conflict.

3 years agoAdding noise diff code.
rtrimana [Mon, 17 Sep 2018 23:27:23 +0000 (16:27 -0700)]
Adding noise diff code.

3 years agoFinished java-based clustering implementation (note: had to move Clusterable implemen...
Janus Varmarken [Mon, 17 Sep 2018 22:39:40 +0000 (15:39 -0700)]
Finished java-based clustering implementation (note: had to move Clusterable implementation to PcapPacketPair and get rid of PcapPacketPairWrapper).

3 years agoPcapPacketPairWrapper: add boolean value for toggling between simplistic and sophisti...
Janus Varmarken [Mon, 17 Sep 2018 20:45:12 +0000 (13:45 -0700)]
PcapPacketPairWrapper: add boolean value for toggling between simplistic and sophisticated source matching

3 years agochanges to .iml file that didn't make it into last commit (rebase action)
Janus Varmarken [Mon, 17 Sep 2018 18:18:58 +0000 (11:18 -0700)]
changes to .iml file that didn't make it into last commit (rebase action)

3 years agoAdd Apache Commons Math as dependency and implement first version of PcapPacketPairWr...
Janus Varmarken [Mon, 17 Sep 2018 18:00:41 +0000 (11:00 -0700)]
Add Apache Commons Math as dependency and implement first version of PcapPacketPairWrapper to allow for clustering in Java

3 years agoNew scripts to cluster based on C->S/S->C direction filter; improving the plot's...
rtrimana [Wed, 12 Sep 2018 22:29:21 +0000 (15:29 -0700)]
New scripts to cluster based on C->S/S->C direction filter; improving the plot's appearance in terms of scaling etc.

3 years agoPrinting directions in the toCSV() method in PrintUtils
rtrimana [Wed, 12 Sep 2018 15:41:38 +0000 (08:41 -0700)]
Printing directions in the toCSV() method in PrintUtils

3 years agoMerge branch 'master' of https://github.uci.edu/rtrimana/smart_home_traffic
rtrimana [Tue, 11 Sep 2018 23:13:32 +0000 (16:13 -0700)]
Merge branch 'master' of https://github.uci.edu/rtrimana/smart_home_traffic

3 years agoCommitting changes for merging.
rtrimana [Tue, 11 Sep 2018 23:13:14 +0000 (16:13 -0700)]
Committing changes for merging.

3 years agoBugfix: fix null-pointer when formatting PcapPacketPairs as CSV
Janus Varmarken [Tue, 11 Sep 2018 23:02:09 +0000 (16:02 -0700)]
Bugfix: fix null-pointer when formatting PcapPacketPairs as CSV

3 years agoAdded support for extracting packet pairs of only TLS Application Data packets (see...
Janus Varmarken [Tue, 11 Sep 2018 00:33:49 +0000 (17:33 -0700)]
Added support for extracting packet pairs of only TLS Application Data packets (see TcpConversationUtils.extractTlsAppDataPacketPairs(Conversation)). Added support for converting PcapPacketPairs to CSV string (see PrintUtils.java).

3 years agoAdding a proof of concept for clustering with source and destination (4 dimensional...
rtrimana [Tue, 11 Sep 2018 00:05:12 +0000 (17:05 -0700)]
Adding a proof of concept for clustering with source and destination (4 dimensional instead of 2 dimensional).

3 years agoFixing scripts and adding a script to create diff plot between two lists of datapoints.
rtrimana [Mon, 10 Sep 2018 22:32:59 +0000 (15:32 -0700)]
Fixing scripts and adding a script to create diff plot between two lists of datapoints.

3 years agoAdding info about TP-Link bulb device IP.
rtrimana [Thu, 6 Sep 2018 23:45:51 +0000 (16:45 -0700)]
Adding info about TP-Link bulb device IP.

3 years agoAdding some changes related to the second batch of experiments to cluster pairs.
rtrimana [Thu, 6 Sep 2018 00:52:37 +0000 (17:52 -0700)]
Adding some changes related to the second batch of experiments to cluster pairs.

3 years agoImproving colors and looks of graph plots.
rtrimana [Wed, 5 Sep 2018 22:21:19 +0000 (15:21 -0700)]
Improving colors and looks of graph plots.

3 years agoMerging changes.
rtrimana [Wed, 5 Sep 2018 21:28:03 +0000 (14:28 -0700)]
Merging changes.

3 years agoAdding pair checking into the pairing process
rtrimana [Wed, 5 Sep 2018 21:25:44 +0000 (14:25 -0700)]
Adding pair checking into the pairing process

3 years agoDo some refactoring to prevent code duplication; change string sequence generation...
Janus Varmarken [Tue, 4 Sep 2018 19:08:21 +0000 (12:08 -0700)]
Do some refactoring to prevent code duplication; change string sequence generation/grouping to utilize Stream API.

3 years agoMerge branch 'master' of https://github.uci.edu/rtrimana/smart_home_traffic
rtrimana [Sat, 1 Sep 2018 00:00:38 +0000 (17:00 -0700)]
Merge branch 'master' of https://github.uci.edu/rtrimana/smart_home_traffic

3 years agoAdding plotting scriptto plot both on and off plots at once.
rtrimana [Sat, 1 Sep 2018 00:00:17 +0000 (17:00 -0700)]
Adding plotting scriptto plot both on and off plots at once.

3 years agoMerging
rtrimana [Fri, 31 Aug 2018 23:31:51 +0000 (16:31 -0700)]
Merging

3 years agoAdding plotting using DBSCAN
rtrimana [Fri, 31 Aug 2018 23:30:15 +0000 (16:30 -0700)]
Adding plotting using DBSCAN

3 years agoConversation.java: added unconvetional TLS ports used by WeMo plugs and LiFX bulbs...
Janus Varmarken [Fri, 31 Aug 2018 22:38:00 +0000 (15:38 -0700)]
Conversation.java: added unconvetional TLS ports used by WeMo plugs and LiFX bulbs to isTls()

3 years agoSome exploratory work on extracting TLS App Data sequence
Janus Varmarken [Fri, 31 Aug 2018 22:21:27 +0000 (15:21 -0700)]
Some exploratory work on extracting TLS App Data sequence

3 years agoTcpConversationUtils.java: add method for grouping conversations by TLS Application...
Janus Varmarken [Fri, 31 Aug 2018 22:18:07 +0000 (15:18 -0700)]
TcpConversationUtils.java: add method for grouping conversations by TLS Application Data sequence; add method for getting packet length sequence for TLS Application data.

3 years agoAdding more checks for Application Data; basically if a packet has the Application...
rtrimana [Thu, 30 Aug 2018 21:37:54 +0000 (14:37 -0700)]
Adding more checks for Application Data; basically if a packet has the Application Data type (=23) and it exceeds the MTU size 1514, then the next packet should be Application Data packet as well.

3 years agoSimple Python script to plot datapoints based on clusters.
rtrimana [Tue, 28 Aug 2018 18:51:04 +0000 (11:51 -0700)]
Simple Python script to plot datapoints based on clusters.

3 years agoExtracting Application Data pairs into files for ON/OFF events.
rtrimana [Tue, 28 Aug 2018 00:02:24 +0000 (17:02 -0700)]
Extracting Application Data pairs into files for ON/OFF events.

3 years agoAdding a boolean variable to choose between the verbose or the concise version of...
rtrimana [Mon, 27 Aug 2018 18:43:19 +0000 (11:43 -0700)]
Adding a boolean variable to choose between the verbose or the concise version of the packet lengths string.

3 years agoConversation.java: implemented logging of TLS application data packets.
Janus Varmarken [Sat, 25 Aug 2018 07:14:50 +0000 (00:14 -0700)]
Conversation.java: implemented logging of TLS application data packets.

3 years agoAdding plotting of points with centroid coordinates and frequencies.
rtrimana [Thu, 23 Aug 2018 23:22:16 +0000 (16:22 -0700)]
Adding plotting of points with centroid coordinates and frequencies.

3 years agoMerge branch 'master' of https://github.uci.edu/rtrimana/smart_home_traffic
rtrimana [Thu, 23 Aug 2018 17:26:54 +0000 (10:26 -0700)]
Merge branch 'master' of https://github.uci.edu/rtrimana/smart_home_traffic

3 years agoAdding Python ML examples of clustering data points.
rtrimana [Thu, 23 Aug 2018 17:26:31 +0000 (10:26 -0700)]
Adding Python ML examples of clustering data points.

3 years agoSequenceExtration.java: finish up naive implementation of extract(...) and move to...
Janus Varmarken [Tue, 21 Aug 2018 22:11:08 +0000 (15:11 -0700)]
SequenceExtration.java: finish up naive implementation of extract(...) and move to seqalignment subpackage.
Main.java: add code for exploring the number of false positives for sequence alignment.
TcpConversationUtils.java: move implementation of getPacketLengthSequence(Conversation) to this class from SequenceExtraction as it is a generally useful util method.

3 years agoupdate to gradle 4.9
Janus Varmarken [Tue, 21 Aug 2018 18:15:27 +0000 (11:15 -0700)]
update to gradle 4.9

3 years agoMerge branch 'master' of https://github.uci.edu/rtrimana/smart_home_traffic
rtrimana [Mon, 20 Aug 2018 18:59:43 +0000 (11:59 -0700)]
Merge branch 'master' of https://github.uci.edu/rtrimana/smart_home_traffic

3 years agoFew more tests in Main.java
rtrimana [Mon, 20 Aug 2018 18:59:22 +0000 (11:59 -0700)]
Few more tests in Main.java

3 years agoSequenceExtraction.java: work in progress, ignore it.
Janus Varmarken [Sun, 19 Aug 2018 06:54:36 +0000 (23:54 -0700)]
SequenceExtraction.java: work in progress, ignore it.

3 years agoTcpConversationUtils.java: make SYNs, FINs, and RSTs part of sequence string produced...
Janus Varmarken [Sun, 19 Aug 2018 06:51:23 +0000 (23:51 -0700)]
TcpConversationUtils.java: make SYNs, FINs, and RSTs part of sequence string produced by groupConversationsByPacketSequence(Collection<Conversation>)