Configuring the Kafka Collector
You can configure Kafka Collector, to collect the data from the various data
sources.
- To configure the Kafka collector, navigate toAdministration>Configuration>Collectors and Connectors.
- From the Collectors section, clickAdd.
- From the Collector Selection page, selectkafka-collector.Thekafka-collectorpage is displayed.You must provide the source Kafka topic where data come from, and the respective mapping that will be used to process incoming records.Following table lists the parameter to configure the Kafka Collector.Input parameterDescriptionDefault ValueName of the collectorProvide name of the collectorNATypeChoose from following:
- Metrics
- Events
MetricsData CentreSelect the location of the collector data.NASource Kafka ConfigurationTopicProvide source Kafka topic where data can come from.metrics: If type of the collector is Metrics.events: If type of the collector is Events.Bootstrap serversProvide source Kafka broker detail.edge-kafka-bootstrap:9093Destination Kafka Configuration (available only if type of the collector is Events.)TopicProvide Kafka topic to which the processed Event data must be written.vsa_events_rawBootstrap serversProvide destination Kafka broker detail.edge-kafka-bootstrap:9093Advanced ConfigurationApplication IdProvide an identifier for the stream processing application. Must be unique within the Kafka cluster.Auto Offset ResetPossible values are:- Earliest: automatically reset the offset to the earliest offset.
- Latest: automatically reset the offset to the latest offset.
- None: throw exception to the consumer if no previous offset is found for the consumer's group.
- anything else: throw exception to the consumer.
latestGroup IDProvide an unique string that identifies the Connect cluster group this worker belongs to.MapperKafka MapperName of the Kafka Mapper to be used with this collector. Drop down list of all configured Kafka Mapper based on the type selected earlier. Refer, Kafka Mapper section for more informationFirst entry of the drop down list.Mapping Definition PreviewPreview the selected mapping schema definition as configured in the Kafka Mapper .NA