Technology sharing

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

missionem details

Negotium contentum est Flume instituere et configurare, ac probare et colligere notitias effusas.

negotium ductu

Flume saepe usus est ut instrumentum ad collectionem real-time datae. Data collectae in HDFS vel nuntio queue Kafka condi possunt.

De gradibus institutionis specificae sunt hae:

1. Unzip Flume compressa sarcina

2. Configure Flume variabilium rerum

3. Lima configurationis Flume Modificare. Fasciculus configurationis Flume in conf in Flume institutionis presul reponitur.

4. Notitia collecta Flume per remotum login

5. Store notitia collecta a Flume in HDFS

Missionis factum

Install Flume 1.

Potes invenire sarcina institutionem in /opt/software/presul, unzip sarcina institutionis effingo sarcina institutionis ad /opt/app directorium

Judicium in master1;

[root@master1 ~]# cd /opt/software/
[root@master1 software]# tar -xzf apache-flume-1.9.0-bin.tar.gz -C /opt/app/

2. Pone Flume in variabilium rerum

File /etc/profile lima et de nuntia Flume iter in domum tuam et viam bin ad iter adde:

export FLUME_HOME=/opt/app/apache-flume-1.9.0-bin
export PATH=$PATH:$FLUME_HOME/bin

Configurationem fasciculi /etc/profile onerare et roborare effectum

[root@master1 ~]# source /etc/profile
[root@master1 ~]# echo $FLUME_HOME

3. constitue de configuratione file flume-env.sh

Effingo et rename flume-env.sh.template ad flume-env.sh sub $FLUME_HOME/conf, ac conf/flume-env.sh conformationem lima mutare

[root@master1 ~]# cd $FLUME_HOME/conf
[root@master1 conf]# cp flume-env.sh.template flume-env.sh
[root@master1 conf]# vi flume-env.sh

Appendice sequentia contenta in fine fasciculi configurationis:

JAVA_HOME=/opt/app/jdk1.8.0_181
JAVA_OPTS="-Xms100m -Xmx200m -Dcom.sun.management.jmxremote"

4. Quin institutionem (telnet)

Configuratio lima mutare flume-conf

Mutare file flume-conf.properties.template in directorio $FLUME_HOME/conf, effingo ac renominando ad flume-conf.properties

[root@master1 ~]# cd $FLUME_HOME/conf
[root@master1 conf]# cp flume-conf.properties.template flume-conf.properties
[root@master1 conf]# vi flume-conf.properties

Mutare contenta in conformatione fasciculi sescarii hoc modo:

# The configuration file needs to define the sources, the channels and the sinks.
# Sources, channels and sinks are defined per agent, in this case called 'a1'
a1.sources = r1
a1.sinks = k1
a1.channels = c1
# For each one of the sources, the type is defined
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = 44444
#The channel can be defined as follows.
a1.sources.r1.channels = c1
# Each sink's type must be defined
a1.sinks.k1.type = logger
#Specify the channel the sink should use
a1.sinks.k1.channel = c1
# Each channel's type is defined.
a1.channels.c1.type = memory
# Other config values specific to each type of channel(sink or source)
# can be defined as well
# In this case, it specifies the capacity of the memory channel
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100

Curre in Fiume institutionem Directory

[root@master1 conf]# cd $FLUME_HOME
[root@master1 apache-flume-1.9.0-bin]# flume-ng agent -c ./conf/ -f ./conf/flume-conf.properties -n a1 -Dflume.root.logger=INFO,console

Aperire alium terminum et hoc mandatum intrare:

[root@master1 ~]# telnet localhost 44444
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

Intra sequentia in terminatio:

Hello
                                   

                    上一篇
                    下一篇