See the file integrity monitoring deployment guide for details. The subscriptions are constructed from the configuration. The osquery/tables/events/linux/file_events.cpp table uses a pubsub subscription and implements a subscriber. There's a list of yet-to-be-implemented uses of the inotify publisher, but a simple example includes querying for every change to /etc/passwd. Let's consider Linux's inotify framework: osquery/events/linux/inotify.cpp implements an osquery publisher. Example: inotifyįilesystem events are the simplest example. For every event the publisher receives it will loop through every Subscription and call fire(const EventContextRef, EventTime) to send the event to the subscriber.
#Osquery events are disabled series
The pubsub runflow is exposed as a publisher setUp(), a series of addSubscription(const SubscriptionRef) by subscribers, a publisher configure(), and finally a new thread scheduled with the publisher's run() static method as the entry point. An osquery event subscriber will send subscriptions to a publisher, save published data, and react to a query by returning appropriate data. The loop or callback introspects on the event and sends it to every appropriate subscriber. The publisher loops on some selected resource or uses operating system APIs to register callbacks. ArchitectureĪn osquery event publisher is a combination of a threaded run loop and event storage abstraction. This saves the most memory and disk usage possible while still allowing flexible scheduling. This allows each subscriber to return the exact window of the schedule and delete buffered events immediately. The subscriber tables can detect they are responding to a schedule and may keep track of the last time the scheduled query has executed. These optimization can be disabled using -events_optimize=false. When scheduling queries that include _events (subscriber-based) tables, additional optimizations are invoked.
![osquery events are disabled osquery events are disabled](https://i.pinimg.com/originals/66/a7/f3/66a7f34b588a73f2830fa6ce71bcd00f.jpg)
If you select from this table every second you will constantly see a window of 1 day's worth of process events. At that point all results will be returned and immediately after, any event that happened time-86400 seconds ago will be deleted. For example: the process_events subscriber will buffer process starts until a query selects from this table. This is set to 1 day by default, this expiration occurs when events are selected from their subscriber table. The buffered events will eventually expire! The -events_expiry flag controls the lifetime of buffered events. Thus the shell cannot be used to explore events buffered by the daemon. The shell does not communicate with the osquery daemon, nor does it use the same RocksDB storage. Furthermore, some of the APIs used by the runloops require super-user privileges or non-default flags and options. If the shell is not running, no events are being buffered. This is because the event loops start and stop with the process. When using the osqueryi shell, these tables will mostly remain empty. These tables are a "query-time" abstraction that allow you to use SQL aggregations and a time column for optimizing lookups. These tables will perform lookups into the osquery backing storage: RocksDB, for events buffered by the subscribers. Query and table usageĪlmost every pubsub-based table ends with a _events or _changes. Otherwise, all of the *_events tables must have a time column, this is used to optimize searching: SELECT * FROM process_events WHERE time > NOW() - 300.
![osquery events are disabled osquery events are disabled](https://www.utoronto.ca/sites/default/files/paralympics_1140.jpg)
If no time range is provided, as in: SELECT * FROM process_events, it is assumed you want to scan from t=[0, now).
![osquery events are disabled osquery events are disabled](http://enablemagazine.co.uk/wp-content/uploads/2018/05/gsc.jpg)
#Osquery events are disabled generator
As events occur, the rows returned by a query will compound, as such selecting from an event-based virtual table generator should always include a time range. The time of event, storage history, and applicable (final) virtual table data information must be carefully considered. This reporting pipeline is much more complicated than typical query-time virtual table generation. To solve this, osquery exposes a pubsub framework for aggregating operating system information asynchronously at event time, storing related event details in the osquery backing store, and performing a lookup to report stored rows query time. Consider the processes table: if a process like ps runs for a fraction of a moment there's no way SELECT * FROM processes will ever include the details.
![osquery events are disabled osquery events are disabled](https://asuevents.asu.edu/sites/default/files/ada-signing.jpg)
So whenever a call selects data from time, e.g., SELECT * FROM time the current time of the call will return.įrom an operating systems perspective, query-time synchronous data retrieval is lossy. For example, the time gets the current time and returns it as a single row. Most of osquery's virtual tables are generated when an SQL statement requests data. The pub-sub evented data framework of osquery