Guardium Big Data Intelligence Release Notes**

V4.1 - TBD

  • Search bar indexes SonarK dashboards instead of “old-style” dashboards

V4.0 - Feb 2019

  • Adding and editing of group members (without overwriting Guardium group members)
  • Group filtering in report builder
  • Bind variables editor for multi runs
  • Allow re-running each multi-run individually
  • Web services upload support
  • Enhanced Web services POST support
  • Allow loading previous version of a pipeline in Studio (for recovery and viewing purposes) and recording the pipeline version on the PDF (for audit purposes)
  • Annex columns in SonarW
  • Annex enrichment
  • Annex stage in Studio
  • Projection and match primitives for annexes in Studio
  • Annex support in SonarK
  • Regex enrichment support in Studio
  • Reconciliation-based enrichment in Studio
  • Pipeline version displayed on PDF footer and job’s BV values in header when using $$LMRM__BVS
  • Support $$LMRM__COMPUTE() to allows computing header/footer values based on bind variables
  • On-demand load of group members rather than upon login
  • Dispatcher support for n in header and footer
  • Additional filters in Justify workflow
  • Persist selection of timeframes in date picker
  • Separate retention for attachments in Open vs Closed Justify workflow states
  • Maintain assignee fields when workflow is closed to allow future searches
  • Only workflow admins can see “all tickets”
  • Add download size to dispatcher log
  • Fine-grained control over what users can see in workflow
  • Show five next firing times of a job in the user’s timezone based on the cron definition
  • $cron operator
  • Dynamic (unlimited) number of bind variables
  • Export to PDF in report viewer
  • Infinite scale for Gateway and for SonarW
  • Extended $awk expressions in SonarW and in Gateway
  • Range support in $lookup in Gateway
  • $ipToNumber, $netMask, and $netCIDR operators
  • Parsing of inner JSON and XML within fields by Gateway
  • Direct-to-Cloud architecture option
  • v1 of DB Security 2.0 SaaS product
  • $cluster aggregation stage
  • $rangeLookup operator
  • $containsRank operator
  • $windowJoin operator
  • New SonarK visualizations - swimlane, search tables, pivot tables, polar plots, radar plots, Vega, region maps and coordinate maps
  • Formulas and computed fields in SonarK
  • Alert building and viewing in SonarK
  • Timelion in SonarK
  • Drill-downs in SonarK
  • Access control for SonarK
  • Export to CSV and PDF from SonarK discover panels
  • Built-in UEBA models for SonarG/GBDI
  • Threats Dashboard
  • Purge log report
  • Storage-used Dashboard
  • Trusted entitities application
  • Multi-stage Web service download
  • Support for Guardium grdapi with SSH keys
  • Web-service upload to SharePoint
  • RDBMS pulls to Gateway
  • Workflow triggers
  • GUI screen access audit trail
  • Batch jobs
  • Aliases and export in discover panels in SonarK dashboards
  • New DB360 Dashboard
  • Support for Ranger entitlements*
  • Tenable / Nessus / Rapid 7 Integration*
  • Support for ElasticSearch audit through security x-pack*
  • Support for ElasticSearch entitlements*
  • Support for S3 audit trail*
  • Support for S3 entitlements*
  • Support for S3 Macie alerts*
  • Support for EMR*
  • Support for MongoDB Atlas*
  • Support for MarkLogic*
  • Support for AWS DynamoDB (control plane requests)*
  • Support for AWS RedShift*
  • New VA dashboards and reports*
  • Periodic model-based outlier detection*
  • Security 360 applications and tooling*
  • VA 360 supports Nessus, Nexpose, Qualys, Guardium and Imperva*
  • Regular expression auto-detection, clustering and enrichment application*
  • UEBA applications and tooling*
  • Account classification and change identification*
  • Support for AWS RDS all variants*
  • CEF, LEEF and JSON format enrichment and forwarding*
  • Streaming analytics in Gateway*
  • GCP services for Pub/Sub and StackDriver*
  • Support for Sharepoint Online and Office 365*

V3.2.1 - Apr 2018

  • Timezone specifier for jobs affects generated CSVs
  • Timezone specifier in preferences used for online CSV and report displays
  • Incremental option in report builder
  • Configurable report batch size between 1,000 and 10,000
  • ILMT tag file for Guardium Big Data Intelligence
  • Facilities for supporting GUI integration in Guardium 10.5
  • Remove report timeline functionality
  • Assignment in hierarchy for Justify workflows allows down-hierarchy
  • Attachments to Justify tickets that are not in final state not deleted by dispatcher
  • Update of POI and PrimeFaces libraries
  • Redacting of incoming data and $awk operator
  • Multi-term visualizations in SonarK
  • New SonarK visualizations
  • $cron operator
  • Improved performance for AWS CloudWatch acquisition
  • Remove auto-refresh option on dashboards
  • Better filters for Justify workflows
  • Justify workflow filters uses global time picker
  • Ability to add a self-contained SonarK URL to the custom reports menu through the report builder
  • Admin can see all users’ reports in report builder to publish to all other users
  • Support null in a filter-out pipeline stage
  • Allow importing a filter-out stage from a spreadsheet
  • Netskope interface
  • Deprecate finder in JSON Studio
  • Deprecate old-style job sign-off and comments

V3.2 - Jan 31, 2018

  • Add sum, min, max and avg to reduce
  • VA360
  • Various platform upgrades and hardening
  • Scheduled job’s assign to roles field is affected by bind variables
  • DCAP Central main page
  • Time picker for GUI can control all reports, dashboards and Studio
  • Support for AWS Cloudwatch
  • Support for AWS Aurora
  • Support for AWS MySQL
  • Support for authentication using a SAML server (tested with Ping)
  • Case insensitive equality operator ($caseEq)
  • CIDR-aware and subnet-aware operators - $inCIDR and $inNetwork
  • MongoDB 3.6 compatibility
  • $iterGroup and $iter operators
  • Head/tail grouping operators
  • Run-length compression for sort tables for sparse data
  • Improved setup for CosmosDB cloud sources
  • Migration to stateless data marts for Guardium systems
  • Cached cloud storage usage stats
  • Compute local storage stats before cloud storage stats
  • Currently running query screen and ability for admin to terminate a query
  • LDAP integration can now set application roles
  • Okta interface
  • Report-level signoff process generates a single Justify ticket even when emails are sent to multiple people (e.g. when a field is used to define who to send the emails to)
  • Web services tail
  • $lookup in SonarGateway for enrichment
  • Support for XML parsing through SonarGateway
  • Deleting a user also deletes all privileges
  • Hover over scheduled job shows it’s name
  • Caseless login when authentication is performed through LDAP
  • Ability to use SonarK Discover without a histogram
  • Ability to set a max run time for a query that comes from SonarK (auto-cancel)
  • New visualization widgets in SonarK (e.g. tag cloud) and better support for existing ones
  • Ability to embed a SonarK dashboard in a main page menu
  • Improved cron editor
  • Support for Cassandra auditing
  • Pause and resume scheduled jobs
  • Ability to run stats on collection from analyzer
  • New cron builder UI
  • Force change of passwords

V3.1 - Nov 10, 2017

  • Deprecate search screen; replaced by SonarK introduced in V3.0
  • Timelines
  • Hardening improvements and infrastructure software upgrades
  • Long session report
  • Change sessions reports to allow for reduce
  • Allow saving/loading of reductions
  • Deprecate sessions active-on report
  • Kafka consumer general availability
  • Add limit to reduce and noise cancelation when combining
  • Re-query in reports undoes reduce
  • IP-to-DNS preference to show both together
  • Pipeline description/annotation
  • Flag for requiring sign-off in job definition and merge between report-level workflow and Justify application
  • New SAGE GUI
  • New user administration GUI
  • Machine Learning option for trusted connections
  • SonarK one-level visualization
  • SonarK noise reduction (and other) buttons
  • SonarK dashboards
  • MongoDB 3.4 compatibility
  • Configuration backed up to Cloud (when using cloud management)
  • Reports backed up to Cloud
  • HDFS storage (Technology Preview)
  • Auditing for CosmosDB
  • Support for Azure Event Hubs
  • CyberArk interface
  • SQL Server native auditing using Windows events and syslog
  • New Justify application
  • Web service endpoint in dispatcher
  • ServiceNow Interface
  • Ability to set auto-reduction when adding a report to the menu
  • Natural -1 order by default
  • Concurrent dispatcher jobs
  • Native auditing for Oracle and SQL Server
  • Native auditing for Cloudera, HortonWorks and MapR (Technology Preview)
  • Multiple cleansing, enrichment and mapping options for SonarGateway
  • Prebuilt workflows for outliers, trusted connections and trusted connection revalidation
  • Ops emails (e.g. missing files and disk utilization alerts) support multiple email addresses (comma delimited)
  • Support for Cassandra auditing (Technology Preview)
  • Button showing optimized pipeline in Studio
  • Risk management applications - Vulnerability management and Sensitive data management

V3.0 - July 18, 2017

  • New GUI theme (partial)
  • SonarK
  • SonarC-based cloud architecture for Guardium Big Data Intelligence (polymorphic cloud storage)
  • Syslog ingestion and parsing
  • GDPR application/engine
  • Various performance optimizations and improvements
  • Fuzzy search capabilities
  • SonarGateway syslog and file interface
  • Support for Oracle on AWS RDS auditing - Technology Preview
  • Support for Oracle native auditing through syslog and through XML audit files
  • Predefined drill-downs
  • CSVs that have a cell length over 16k will be split to multiple lines
  • Tailing of an RDBMS table through dispatcher RDBMS jobs
  • Email-to field in dispatcher jobs
  • Scheduled remote pulls for sonargd
  • Trusted Connections (TCs) as part of the SAGE profiling engine
  • Monitoring gap analytics
  • Multi-selection parameter in dashboard variables
  • Edit button on dashboard frames for pipelines and limit fields in dashboard building
  • Hashing signatures for proven non-repudiation and chain-of-custody
  • Guardium Big Data Intelligence Integrity Service
  • Kafka producer
  • Noise reduction in SonarK and in any report
  • Support option in scheduler for delivery of encrypted content to Guardium Big Data Intelligence

V2.8 - April 10, 2017

  • Support for Azure SQL - Technology Preview
  • Projection and match editors are code-sensitive, complete brackets etc.
  • Syslog management from SAGE & templates for CEF/LEEF/RSA (Guardium events)
  • Heatmap improvements
  • Sliding window analytics as reports (in addition to alerts)
  • Policy analysis heatmap for Full SQL source
  • Policy analysis sliding window analysis
  • Session/Query/Exceptions summary/details dashboards
  • Download CSV from Analytics Pipeline Builder
  • IP/DNS auto-completion in ETL layer
  • Faster group members in multi-CM scenarios
  • Improved handling of highly-fragmented data

V2.7 - Feb 10, 2017

  • Support for DMv2
  • Ability to deliver both PDF and CSV in the same email
  • Enhanced CSV dialect support in misc, support for gzip and zipped CSVs
  • Drag and drop reorder for sort, project and joins with a single collection
  • Agents dashboard
  • Failed login and SQL errors top offenders reports
  • User cluster report
  • Redaction operator (replacing any regex match with any string)

V2.6 - Jan 9, 2017

  • Moving average analytics in SAGE
  • Support for FAM activity
  • Support for arbitrary dropped CSVs in ETL (non-Guardium DM extracts)

V2.5 - Dec 5, 2016

  • Search report, pipeline and schedule
  • New ETL and upgrade to column store
  • Various improvements to HADR
  • Various improvements to dispatcher
  • IP-to-hostname resolution in both ETL and GUI
  • Filter-out pipeline operator
  • Verb and Object match operators
  • Justification application
  • Justification and review reports
  • Application-level security
  • Various new operational alerts
  • File pivot reports
  • Heatmap color range not using white

V2.2 - Sept 30, 2016

  • Data level security
  • Field level security
  • DB360 engine & dashboards
  • Datasource management
  • Custom Reports
  • Option for reports that can run disconnected from the Internet
  • Justification application (Beta)
  • LDAP Data Integrator
  • SSH Invocation Integrator
  • RDBMS Data Integrator
  • Multiple new DM imports
  • Guardium Big Data Intelligence Source as global attribute

V2.1 - July 8, 2016

  • S-TAP uptime data in Collector Dashboard
  • Clustering outlier visualizations
  • Clustering algorithms for user classification
  • Group viewing in JSON Studio
  • Discovery data as predefined DM
  • Limit control in forms
  • Local installers in addition to repo access
  • CSV retrieval and ingestion for external systems through SonarDispatcher
  • Beta: DB360 engine & dashboards
  • Beta: Datasource management
  • Updated UI
  • Speed improvements to search application and predefined queries
  • VA and Classifier predefined reports

V2.0 - May 31, 2016

  • Machine learning subsystem for session data, exception data and violation data
  • Noise reduction subsystem
  • Outlier detection
  • OLAP capabilities and window functions for any data domain
  • Support for Guardium 10.1
  • Classifier data as predefined DM
  • Objects/Verbs added to policy violations (support for detailed policy violations DM inserted into policy_violations collection)
  • Runtime report and graph

V1.3 - Feb. 29, 2016

  • User management GUI replaces use of the shell
  • Support for multi-CM environment transparently
  • New group-related search operators
  • Split predefined session reports to active vs opened
  • Predefined reports initialize dates to 1 day

V1.2 - Jan. 25, 2016

  • Security Operations Center (SOC) dashboard
  • Increased throughput per node
  • VA as a predefined DM
  • Snif Buf Usage as a predefined DM
  • Support for type modifications in ETL
  • Session profiling subsystem
  • DM extraction logs as predefined DM for reconciliation capabilities
  • Support for Tableau, Qlik and other BI tools
  • SUpport for Splunk access to Guardium Big Data Intelligence

V1.1 - Nov. 16, 2015

  • Enhanced predefined reports
  • Operations dashboard
  • Support for additional DM extractions (beyond built-in)
  • Support for Guardium V10 (including outliers)
  • $contains and $sontainsTuples operators for fast matching of query strings based on groups
  • User sign-of and workflow processes
  • Alerts for errors and exception conditions
  • Added visualizations such an heatmaps, gauges, bullets, punchcards and more

V1.0 - Sept. 14, 2015

  • First GA release of Guardium Big Data Intelligence

** Release notes are for GBDI, SonarG and DCAP Central since we release the SonarC platform and all products at the same times. * DCAP Central only

Known Issues

  • SNRG 2298: The view pipeline code and validate function will omit a space when the field have is not quoted. This has no effect on the pipeline being run, just the code viewer.
  • SNRG 2276: If deleting a custom menu item and that menu item is the last one selected on the home page, then a validation error will occur on the next submit. This error is benign and can be ignored, and will not longer show after the firt time.
  • SNRG2201: In order to compensate for usage of database name in most database types and service name or sid in Oracle, service name in SAGE data is either the Oracle service name / SID or the database name in other platforms. DB360 and profiling data and jobs (specifically the ae_dt_session job) will copy the database name automatically into the Service Name field. More detail below.
  • SNRG2150: Due to the meaning of as an escape character, whenever you do a query and need to use a you must also escape it. If for example you want to query on a DB User Name ENCORE\JANE you must use ENCORE\JANE. The displays show the data as ENCORE\JANE - just the queries need to use \.
  • SNRG2142: If you invoke an API and check the “remember me” checkbox and then navigate to the Guardium Big Data Intelligence home page, the applications will still be diabled. You must be logged in through the Guardium Big Data Intelligence login page to enable all Guardium Big Data Intelligence apps. Logout and then login on the Guardium Big Data Intelligence home page.
  • SNRG2099: % cannot be used as a character in parameter names when scheduling a report.
  • SNRG2096: Residual table formatter possible from dashboard reports when using the Studio (Analyze) application. Delete the formatter if you get an error message while rendering a report in the Studio. Report results are correct - this is just a formatting warning.
  • SNRG2011: All parameters of a report must be filled in; Users should not delete parameters or the system will not be able to resolve parameters entered in various stages. Workaround: Close the form and Submit again on the main screen to get the form with all parameters and their default values.
  • SNRG1971: __ae_pr_alert_syslog and __ae_outlier_syslog fire every 5 minutes instead of every 20 minutes
  • SNRG1920: Profile alerts for last time period repeated.
  • SNRG1862: Scheduled jobs may be deleted from the “scheduled jobs” links but should not be modified unless the URL is recomputed.
  • SNRG1802: SAGE profiling email is missing a subject
  • SNRG1334: Scheduling admin reports can only be done if the appropriate section in dispatcher.conf is changed to use an admin-role credential (e.g. in the lmrm__scheduler section).
  • SNRG1072: All browser tabs open within the same browser share a single Web session. It is therefore not recommended to be logged in using one application in Guardium Big Data Intelligence and try to use a different application in the same browser nor to be logged in as two distinct users. One session will disable the other.
  • SNRG952: Catalina and sonard logs show Authorization failure - this is not a real failure but rather due to the mongodb driver producing a “show collections” on the admin database. A workaround can be to add that privilege to the users, although this message is benign and can be ignored.
  • SNRG864: When switching from a single group_member environment (since CM) to multiple, group members may be void for one hour.
  • SNRG617: When first enabling DM extractions on Guardium collectors the first hour’s extract may be empty (depending on when it was scheduled vs. when it was enabled).
  • SNRG489: Usage of Internet Explorer is discouraged.
  • SNRG210: Strings entered as parameters (not as a regex) must be double quoted.

Additional known issues related to SonarW, SonarSQL and JSON Studio apply.