FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Cluster Development

 
 
LinkBack Thread Tools
 
Old 10-05-2011, 08:56 AM
Kazunori INOUE
 
Default backend plugin for monitoring a host's status

Hi all,

I think that the communication function of fence-virt is flexible,
so I want to use it more effectively.

Therefore I made backend plugin for a guest to get the host's status
using the communication facility of fence-virt,
and I changed to allow specifying one more backend (for fencing, and
replying the host's status).

I created the backend "pm-monitor" which has met the following
configurations / requirements.
- Both hosts and VMs, cluster (Pacemaker) have been configured.

Here's an overview of function. Please refer to attached 'overview.png'.
(*) pingd resource notifies the status of connection with a specific
host to pacemaker, and pacemaker manages the result.
(1) resource (vm-client) which requires the host's status is executed.
(2) vm-client requests 'host_status (result of pingd)' to the host
with fence_virt.
(3) use the serial listener,
(4) fence_virtd (pm-monitor backend) gets the 'result of pingd' from
pacemaker and answers it after conversion.
- the conversion rule is set in /etc/pm-monitor.conf

Here's a description of the attached files.
* add_general_backend.patch
- add the server/pm-fence.c
- change the configure.in and server/Makefile.in
* overview.png
- figure of the overview.
* vm-client
- Resource Agent for VM.
- I'm going to post this RA to the appropriate community
(https://github.com/ClusterLabs/resource-agents).
* fence_virt.conf
- sample configuration.
* pm-monitor.conf
- sample configuration for pm-monitor.so

* host.cli, mon-host.txt
- sample configuration file of a host cluster,
and mon-host.txt is an output of the crm_mon command.
* VM.cli, mon-VM.txt
- sample configuration file of a VM cluster,
and mon-VM.txt is an output of the crm_mon command.

Best Regards
diff -urN fence-virt-200eab4/client/main.c mod/client/main.c
--- fence-virt-200eab4/client/main.c 2011-08-11 09:53:47.000000000 +0900
+++ mod/client/main.c 2011-09-20 10:40:23.125557546 +0900
@@ -56,10 +56,10 @@

args_init(&args);
if (!strcmp(basename(argv[0]), "fence_xvm")) {
- my_options = "di:a:r:C:c:k:M:H:uo:t:?hV";
+ my_options = "di:a:r:C:c:k:M:H:uo:t:Q:?hV";
args.mode = MODE_MULTICAST;
} else {
- my_options = "dD:P:A:M:H:t:?hV";
+ my_options = "dD:P:A:M:H:t:Q:?hV";
args.mode = MODE_SERIAL;
}

@@ -100,10 +100,15 @@
/* Additional validation here */
if (!args.domain && (args.op != FENCE_DEVSTATUS &&
args.op != FENCE_HOSTLIST &&
- args.op != FENCE_METADATA)) {
+ args.op != FENCE_METADATA &&
+ args.op != MONITOR_HOSTSTATUS)) {
printf("No domain specified!
");
args.flags |= F_ERR;
}
+ if (!args.query && args.op == MONITOR_HOSTSTATUS) {
+ printf("No query(host_status) specified!
");
+ args.flags |= F_ERR;
+ }

if (args.flags & F_ERR) {
args_usage(argv[0], my_options, (argc == 1));
@@ -136,6 +141,9 @@
case RESP_PERM:
printf("Permission denied
");
break;
+ case RESP_NOTSUPPORT:
+ printf("Operation not supported
");
+ break;
default:
printf("Unknown response (%d)
", ret);
break;
diff -urN fence-virt-200eab4/client/mcast.c mod/client/mcast.c
--- fence-virt-200eab4/client/mcast.c 2011-09-20 10:17:19.012256692 +0900
+++ mod/client/mcast.c 2011-09-20 10:40:23.125557546 +0900
@@ -124,6 +124,50 @@
}


+void
+do_read_host_status(int fd, int timeout)
+{
+ host_status_t hstatus;
+ fd_set rfds;
+ struct timeval tv;
+ int ret;
+ int flag = 0;
+
+ do {
+ FD_ZERO(&rfds);
+ FD_SET(fd, &rfds);
+ tv.tv_sec = timeout;
+ tv.tv_usec = 0;
+
+ ret = _select_retry(fd+1, &rfds, NULL, NULL, &tv);
+ if (ret == 0) {
+ printf("Timed out!
");
+ break;
+ }
+
+ ret = _read_retry(fd, &hstatus, sizeof(hstatus), &tv);
+ if (ret < sizeof(hstatus)) {
+ printf("Bad read!
");
+ break;
+ }
+
+ if (strlen((char *)hstatus.req) == 0 &&
+ strlen((char *)hstatus.res) == 0)
+ break;
+
+ if (flag)
+ printf(",");
+ else
+ flag = 1;
+ printf("%s=%s", hstatus.req, hstatus.res);
+
+ } while (1);
+
+ if (flag)
+ printf("
");
+}
+
+
static int
tcp_exchange(int fd, fence_auth_type_t auth, void *key,
size_t key_len, int timeout)
diff -urN fence-virt-200eab4/client/options.c mod/client/options.c
--- fence-virt-200eab4/client/options.c 2011-09-12 10:41:16.000000000 +0900
+++ mod/client/options.c 2011-09-20 10:40:23.126631731 +0900
@@ -236,6 +236,8 @@
args->op = FENCE_HOSTLIST;
} else if (!strcasecmp(value, "metadata")) {
args->op = FENCE_METADATA;
+ } else if (!strcasecmp(value, "host_status")) {
+ args->op = MONITOR_HOSTSTATUS;
} else {
printf("Unsupported operation: %s
", value);
args->flags |= F_ERR;
@@ -346,6 +348,34 @@
}


+static inline void
+assign_query(fence_virt_args_t *args, struct arg_info *arg, char *value)
+{
+ if (!value)
+ return;
+
+ if (args->query) {
+ printf("query may not be specified more than once
");
+ args->flags |= F_ERR;
+ return;
+ }
+
+ args->query = strdup(value);
+
+ if (strlen(value) <= 0) {
+ printf("Invalid query
");
+ args->flags |= F_ERR;
+ }
+
+ if (strlen(value) >= MAX_QUERY_LENGTH) {
+ errno = ENAMETOOLONG;
+ printf("Invalid query: '%s' (%s)
",
+ value, strerror(errno));
+ args->flags |= F_ERR;
+ }
+}
+
+
static void
print_desc_xml(const char *desc)
{
@@ -474,6 +504,11 @@
"Fencing timeout (in seconds; default=30)",
assign_timeout },

+ { 'Q', "-Q <query>", "query",
+ 0, "string", NULL,
+ "query",
+ assign_query },
+
{ 'h', "-h", NULL,
0, "boolean", "0",
"Help",
@@ -541,6 +576,7 @@
{
args->domain = NULL;
//args->uri = NULL;
+ args->query = NULL;
args->op = FENCE_REBOOT;
args->net.key_file = strdup(DEFAULT_KEY_FILE);
args->net.hash = DEFAULT_HASH;
@@ -574,6 +610,7 @@
printf("-- args @ %p --
", args);
_pr_str(args->domain);
_pr_int(args->op);
+ _pr_str(args->query);

_pr_str(args->net.key_file);
_pr_int(args->net.hash);
diff -urN fence-virt-200eab4/client/serial.c mod/client/serial.c
--- fence-virt-200eab4/client/serial.c 2011-08-11 09:53:47.000000000 +0900
+++ mod/client/serial.c 2011-09-20 10:40:23.126631731 +0900
@@ -180,6 +180,7 @@
}

void do_read_hostlist(int fd, int timeout);
+void do_read_host_status(int fd, int timeout);

int
wait_for(int fd, const char *pattern, size_t size, struct timeval *tout)
@@ -264,6 +265,8 @@

if (args->domain)
strncpy((char *)req.domain, args->domain, sizeof(req.domain));
+ if (args->query)
+ strncpy((char *)req.query, args->query, sizeof(req.query));

tv.tv_sec = 3;
tv.tv_usec = 0;
@@ -295,6 +298,11 @@
do_read_hostlist(fd, args->timeout);
ret = 0;
}
+ if (resp.response == RESP_HOSTSTATUS) {
+ /* ok read host_status */
+ do_read_host_status(fd, args->timeout);
+ ret = 0;
+ }

close(fd);

diff -urN fence-virt-200eab4/configure.in mod/configure.in
--- fence-virt-200eab4/configure.in 2011-08-11 09:53:47.000000000 +0900
+++ mod/configure.in 2011-09-20 10:40:23.127631217 +0900
@@ -26,6 +26,8 @@
AC_CHECK_LIB([virt], [virConnectOpen])
AC_CHECK_LIB([xml2], [main])

+AC_SEARCH_LIBS([read_attr_delegate], [cib], [], [ pm_ver=1.0 ])
+AC_SUBST(pm_ver)

# Checks for header files.
AC_HEADER_DIRENT
@@ -92,6 +94,13 @@
[ mod_libvirt_qpid=$enableval ], [ mod_libvirt_qpid=yes ])
AC_SUBST(mod_libvirt_qpid)

+# pm-monitor plugin: Disabled by default
+AC_ARG_ENABLE(pm-monitor-plugin,
+[AS_HELP_STRING([--enable-pm-monitor-plugin],
+ [Enable pm-monitor backend plugin])],
+[ mod_pm_monitor=$enableval ], [ mod_pm_monitor=no ])
+AC_SUBST(mod_pm_monitor)
+
# multicast plugin: Enabled by default
AC_ARG_ENABLE(multicast-plugin,
[AS_HELP_STRING([--disable-multicast-plugin],
diff -urN fence-virt-200eab4/include/options.h mod/include/options.h
--- fence-virt-200eab4/include/options.h 2011-08-11 09:53:47.000000000 +0900
+++ mod/include/options.h 2011-09-20 10:40:23.127631217 +0900
@@ -40,6 +40,7 @@

typedef struct {
char *domain;
+ char *query;
fence_cmd_t op;
client_mode_t mode;
int debug;
diff -urN fence-virt-200eab4/include/server_plugin.h mod/include/server_plugin.h
--- fence-virt-200eab4/include/server_plugin.h 2011-08-11 09:53:47.000000000 +0900
+++ mod/include/server_plugin.h 2011-09-20 10:40:23.128592732 +0900
@@ -56,6 +56,10 @@
typedef int (*fence_hostlist_callback)(hostlist_callback cb,
void *arg, void *priv);

+typedef int (*hoststatus_callback)(const char *req, const char *res, int fd);
+typedef int (*monitor_hoststatus_callback)(hoststatus_callback cb,
+ int fd, const char *req, void *priv);
+
typedef int (*backend_init_fn)(backend_context_t *c,
config_object_t *config);
typedef int (*backend_cleanup_fn)(backend_context_t c);
@@ -68,6 +72,7 @@
fence_status_callback status;
fence_devstatus_callback devstatus;
fence_hostlist_callback hostlist;
+ monitor_hoststatus_callback hoststatus;
} fence_callbacks_t;

typedef struct backend_plugin {
@@ -81,9 +86,11 @@

typedef int (*listener_init_fn)(listener_context_t *c,
const fence_callbacks_t *cb,
+ const fence_callbacks_t *cb2,
config_object_t *config,
map_object_t *map,
- void *priv);
+ void *priv,
+ void *priv2);
typedef int (*listener_dispatch_fn)(listener_context_t c,
struct timeval *timeout);
typedef int (*listener_cleanup_fn)(listener_context_t c);
diff -urN fence-virt-200eab4/include/xvm.h mod/include/xvm.h
--- fence-virt-200eab4/include/xvm.h 2011-08-11 09:53:47.000000000 +0900
+++ mod/include/xvm.h 2011-09-20 10:40:23.128592732 +0900
@@ -31,6 +31,7 @@
#define MAX_ADDR_LEN sizeof(struct sockaddr_in6)
#define DOMAIN0NAME "Domain-0"
#define DOMAIN0UUID "00000000-0000-0000-0000-000000000000"
+#define MAX_QUERY_LENGTH 512

typedef enum {
HASH_NONE = 0x0, /* No packet signing */
@@ -59,7 +60,8 @@
FENCE_STATUS = 0x4, /* virtual machine status (off/on) */
FENCE_DEVSTATUS = 0x5, /* Status of the fencing device */
FENCE_HOSTLIST = 0x6, /* List VMs controllable */
- FENCE_METADATA = 0x7
+ FENCE_METADATA = 0x7,
+ MONITOR_HOSTSTATUS = 0x8
} fence_cmd_t;

#define DEFAULT_TTL 4
@@ -107,6 +109,10 @@
uint8_t pad;
} host_state_t;

+typedef struct __attribute__ ((packed)) _host_status {
+ uint8_t req[128];
+ uint8_t res[128];
+} host_status_t;

#define DEFAULT_SERIAL_DEVICE "/dev/ttyS1"
#define DEFAULT_SERIAL_SPEED "115200,8N1"
@@ -119,6 +125,7 @@
uint8_t flags;
uint8_t domain[MAX_DOMAINNAME_LENGTH];
uint32_t seqno;
+ uint8_t query[MAX_QUERY_LENGTH];
} serial_req_t;

#if __BYTE_ORDER == __BIG_ENDIAN
@@ -151,6 +158,8 @@
#define RESP_FAIL 1
#define RESP_OFF 2
#define RESP_PERM 3
+#define RESP_NOTSUPPORT 4
+#define RESP_HOSTSTATUS 252
#define RESP_HOSTLIST 253


diff -urN fence-virt-200eab4/server/Makefile.in mod/server/Makefile.in
--- fence-virt-200eab4/server/Makefile.in 2011-08-11 09:53:47.000000000 +0900
+++ mod/server/Makefile.in 2011-09-20 10:40:23.129509700 +0900
@@ -22,6 +22,8 @@
MAIN_LIBS=-L../config -lsimpleconfig -ldl
AIS_LIBS=-L/usr/lib64/openais -lSaCkpt
COROSYNC_LIBS=-L/usr/lib64/corosync -lcpg
+PACEMAKER_LIBS=-lcib -lpe_status -lncurses
+PACEMAKER_INCLUDES=-I/usr/include/glib-2.0 -I$(libdir)/glib-2.0/include -I/usr/include/pacemaker -I/usr/include/heartbeat
CMAN_LIBS=-lcman
VIRT_LIBS=-lvirt
VIRT_QPID=-lqmf2 -lqpidclient -lqpidtypes -lqpidcommon -lqpidmessaging
@@ -42,6 +44,7 @@
null_so_SOURCES = null.c
libvirt_qpid_so_SOURCES = uuid-test.c
libvirt_qpid_cxx_so_SOURCES = libvirt-qpid.cpp
+pm_monitor_so_SOURCES = pm-monitor.c
multicast_so_SOURCES = mcast.c history.c
checkpoint_so_SOURCES = virt.c vm_states.c history.c checkpoint.c cpg.c
serial_so_SOURCES = virt-serial.c virt-sockets.c serial.c history.c
@@ -54,6 +57,7 @@
mod_libvirt=@mod_libvirt@
mod_checkpoint=@mod_checkpoint@
mod_libvirt_qpid=@mod_libvirt_qpid@
+mod_pm_monitor=@mod_pm_monitor@
mod_multicast=@mod_multicast@
mod_serial=@mod_serial@

@@ -71,6 +75,9 @@
ifneq ($(mod_libvirt_qpid),no)
MODULES+=libvirt-qpid.so
endif
+ifneq ($(mod_pm_monitor),no)
+MODULES+=pm-monitor.so
+endif
ifneq ($(mod_multicast),no)
MODULES+=multicast.so
endif
@@ -100,6 +107,10 @@
fence_virtd_cxx_SOURCES+=${libvirt_qpid_cxx_so_SOU RCES}
LIBS+=$(VIRT_QPID)
endif
+ifneq ($(mod_pm_monitor),no)
+fence_virtd_SOURCES+=${pm_monitor_so_SOURCES}
+LIBS+=$(PACEMAKER_LIBS)
+endif
ifneq ($(mod_multicast),no)
fence_virtd_SOURCES+=${multicast_so_SOURCES}
LIBS+=$(AIS_LIBS) $(NSS_LIBS)
@@ -116,6 +127,11 @@

CFLAGS+=-DSYSCONFDIR="@sysconfdir@"

+pm_ver=@pm_ver@
+ifeq ($(pm_ver),1.0)
+CFLAGS+=-DPM_1_0
+endif
+
all: ${TARGETS} ${MODULES}

fence_virtd: ${fence_virtd_SOURCES:.c=.o} ${fence_virtd_cxx_SOURCES:.cpp=.opp}
@@ -130,6 +146,9 @@
libvirt-qpid.so: ${libvirt_qpid_so_SOURCES:.c=.o} ${libvirt_qpid_cxx_so_SOURCES:.cpp=.opp}
$(CXX) -o $@ $^ $(LIBS) -shared $(VIRT_QPID)

+pm-monitor.so: ${pm_monitor_so_SOURCES:.c=.o}
+ $(CC) -o $@ $^ $(LIBS) -shared $(PACEMAKER_LIBS)
+
null.so: ${null_so_SOURCES:.c=.o}
$(CC) -o $@ $^ $(LIBS) -shared

@@ -143,6 +162,9 @@
%.o: %.c
$(CC) $(CFLAGS) -c -o $@ $^ $(INCLUDES)

+pm-monitor.o: pm-monitor.c
+ $(CC) $(CFLAGS) -c -o $@ $^ $(INCLUDES) $(PACEMAKER_INCLUDES)
+
%.opp: %.cpp
$(CXX) $(CFLAGS) -c -o $@ $^ $(INCLUDES)

diff -urN fence-virt-200eab4/server/main.c mod/server/main.c
--- fence-virt-200eab4/server/main.c 2011-08-11 09:53:47.000000000 +0900
+++ mod/server/main.c 2011-09-20 10:40:23.129509700 +0900
@@ -46,13 +46,16 @@
char val[4096];
char listener_name[80];
char backend_name[80];
+ char gbackend_name[80];
const char *config_file = DEFAULT_CONFIG_FILE;
config_object_t *config = NULL;
map_object_t *map = NULL;
const listener_plugin_t *lp;
- const backend_plugin_t *p;
+ const backend_plugin_t *p = NULL;
+ const backend_plugin_t *p2 = NULL;
listener_context_t listener_ctx = NULL;
backend_context_t backend_ctx = NULL;
+ backend_context_t gbackend_ctx = NULL;
int debug_set = 0, foreground = 0, wait_for_backend = 0;
int opt, configure = 0;

@@ -129,15 +132,29 @@
if (dget() > 3)
sc_dump(config, stdout);

+ memset(backend_name, 0, sizeof(backend_name));
if (sc_get(config, "fence_virtd/@backend", backend_name,
- sizeof(backend_name))) {
+ sizeof(backend_name)) < 0) {
printf("Failed to determine backend.
");
- printf("%s
", val);
return -1;
}

dbg_printf(1, "Backend plugin: %s
", backend_name);

+ memset(gbackend_name, 0, sizeof(gbackend_name));
+ if (sc_get(config, "fence_virtd/@general_backend", gbackend_name,
+ sizeof(gbackend_name)) < 0) {
+ printf("Failed to determine general_backend.
");
+ return -1;
+ }
+
+ dbg_printf(1, "general_backend plugin: %s
", gbackend_name);
+
+ if (!backend_name[0] && !gbackend_name[0]) {
+ printf("Failed to determine backend.
");
+ return -1;
+ }
+
if (sc_get(config, "fence_virtd/@listener", listener_name,
sizeof(listener_name))) {
printf("Failed to determine backend.
");
@@ -178,10 +195,20 @@
return 1;
}

- p = plugin_find_backend(backend_name);
- if (!p) {
- printf("Could not find backend "%s"
", backend_name);
- return 1;
+ if (backend_name[0]) {
+ p = plugin_find_backend(backend_name);
+ if (!p) {
+ printf("Could not find backend "%s"
", backend_name);
+ return 1;
+ }
+ }
+
+ if (gbackend_name[0]) {
+ p2 = plugin_find_backend(gbackend_name);
+ if (!p2) {
+ printf("Could not find general_backend "%s"
", gbackend_name);
+ return 1;
+ }
}

daemon_init(basename(argv[0]), foreground);
@@ -189,7 +216,7 @@
signal(SIGTERM, exit_handler);
signal(SIGQUIT, exit_handler);

- while (p->init(&backend_ctx, config) < 0) {
+ while (p && p->init(&backend_ctx, config) < 0) {
if (!wait_for_backend) {
if (foreground) {
printf("Backend plugin %s failed to initialize
",
@@ -203,13 +230,27 @@
sleep(5);
}

+ while (p2 && p2->init(&gbackend_ctx, config) < 0) {
+ if (!wait_for_backend) {
+ if (foreground) {
+ printf("general_backend plugin %s failed to initialize
",
+ gbackend_name);
+ }
+ syslog(LOG_ERR,
+ "general_backend plugin %s failed to initialize
",
+ gbackend_name);
+ return 1;
+ }
+ sleep(5);
+ }
+
if (map_load(map, config) < 0) {
syslog(LOG_WARNING, "Failed to load static maps
");
}

/* only client we have now is mcast (fence_xvm behavior) */
- if (lp->init(&listener_ctx, p->callbacks, config, map,
- backend_ctx) < 0) {
+ if (lp->init(&listener_ctx, (p ? p->callbacks : NULL), (p2 ? p2->callbacks : NULL),
+ config, map, backend_ctx, gbackend_ctx) < 0) {
if (foreground) {
printf("Listener plugin %s failed to initialize
",
listener_name);
@@ -226,7 +267,8 @@
sc_release(config);

lp->cleanup(listener_ctx);
- p->cleanup(backend_ctx);
+ if (p) p->cleanup(backend_ctx);
+ if (p2) p2->cleanup(gbackend_ctx);

daemon_cleanup();

diff -urN fence-virt-200eab4/server/mcast.c mod/server/mcast.c
--- fence-virt-200eab4/server/mcast.c 2011-09-20 10:17:15.077381596 +0900
+++ mod/server/mcast.c 2011-09-20 10:40:23.129509700 +0900
@@ -83,11 +83,13 @@
typedef struct _mcast_info {
uint64_t magic;
void *priv;
+ void *priv2;
map_object_t *map;
history_info_t *history;
char key[MAX_KEY_LEN];
mcast_options args;
const fence_callbacks_t *cb;
+ const fence_callbacks_t *cb2;
ssize_t key_len;
int mc_sock;
int need_kill;
@@ -263,7 +265,10 @@

switch(req->request) {
case FENCE_NULL:
- response = info->cb->null((char *)req->domain, info->priv);
+ if (info->cb)
+ response = info->cb->null((char *)req->domain, info->priv);
+ else
+ response = RESP_NOTSUPPORT;
break;
case FENCE_ON:
if (map_check(info->map, ip_addr_src,
@@ -271,8 +276,12 @@
response = RESP_PERM;
break;
}
- response = info->cb->on((char *)req->domain, ip_addr_src,
+
+ if (info->cb)
+ response = info->cb->on((char *)req->domain, ip_addr_src,
req->seqno, info->priv);
+ else
+ response = RESP_NOTSUPPORT;
break;
case FENCE_OFF:
if (map_check(info->map, ip_addr_src,
@@ -280,8 +289,12 @@
response = RESP_PERM;
break;
}
- response = info->cb->off((char *)req->domain, ip_addr_src,
+
+ if (info->cb)
+ response = info->cb->off((char *)req->domain, ip_addr_src,
req->seqno, info->priv);
+ else
+ response = RESP_NOTSUPPORT;
break;
case FENCE_REBOOT:
if (map_check(info->map, ip_addr_src,
@@ -289,19 +302,24 @@
response = RESP_PERM;
break;
}
- response = info->cb->reboot((char *)req->domain, ip_addr_src,
+
+ if (info->cb)
+ response = info->cb->reboot((char *)req->domain, ip_addr_src,
req->seqno, info->priv);
+ else
+ response = RESP_NOTSUPPORT;
break;
case FENCE_STATUS:
- if (map_check(info->map, ip_addr_src,
- (const char *)req->domain) == 0) {
- response = RESP_PERM;
- break;
- }
- response = info->cb->status((char *)req->domain, info->priv);
+ if (info->cb)
+ response = info->cb->status((char *)req->domain, info->priv);
+ else
+ response = RESP_NOTSUPPORT;
break;
case FENCE_DEVSTATUS:
- response = info->cb->devstatus(info->priv);
+ if (info->cb)
+ response = info->cb->devstatus(info->priv);
+ else
+ response = RESP_NOTSUPPORT;
break;
case FENCE_HOSTLIST:
arg.map = info->map;
@@ -309,8 +327,11 @@
arg.fd = fd;

mcast_hostlist_begin(arg.fd);
- response = info->cb->hostlist(mcast_hostlist, &arg,
+ if (info->cb)
+ response = info->cb->hostlist(mcast_hostlist, &arg,
info->priv);
+ else
+ response = RESP_NOTSUPPORT;
mcast_hostlist_end(arg.fd);
break;
}
@@ -516,8 +537,8 @@


static int
-mcast_init(listener_context_t *c, const fence_callbacks_t *cb,
- config_object_t *config, map_object_t *map, void *priv)
+mcast_init(listener_context_t *c, const fence_callbacks_t *cb, const fence_callbacks_t *cb2,
+ config_object_t *config, map_object_t *map, void *priv, void *priv2)
{
mcast_info *info;
int mc_sock, ret;
@@ -536,6 +557,8 @@

info->priv = priv;
info->cb = cb;
+ info->priv2 = priv2;
+ info->cb2 = cb2;
info->map = map;

ret = mcast_config(config, &info->args);
diff -urN fence-virt-200eab4/server/pm-monitor.c mod/server/pm-monitor.c
--- fence-virt-200eab4/server/pm-monitor.c 1970-01-01 09:00:00.000000000 +0900
+++ mod/server/pm-monitor.c 2011-09-20 10:40:23.130631668 +0900
@@ -0,0 +1,620 @@
+/*
+ Copyright Red Hat, Inc. 2006
+
+ This program is free software; you can redistribute it and/or modify it
+ under the terms of the GNU General Public License as published by the
+ Free Software Foundation; either version 2, or (at your option) any
+ later version.
+
+ This program is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; see the file COPYING. If not, write to the
+ Free Software Foundation, Inc., 675 Mass Ave, Cambridge,
+ MA 02139, USA.
+*/
+#include <stdio.h>
+#include <netdb.h>
+#include <errno.h>
+#include <syslog.h>
+#include <simpleconfig.h>
+#include <static_map.h>
+
+#include <server_plugin.h>
+
+#include <sys/utsname.h>
+#include <crm/cib.h>
+#include <crm/pengine/status.h>
+
+/* Local includes */
+#include "xvm.h"
+#include "debug.h"
+
+
+#define BACKEND_NAME "pm-monitor"
+#define VERSION "0.1"
+
+#define MAGIC 0x1e0d197b
+
+#define CONFIG_FILE "/etc/pm-monitor.conf"
+
+enum expr_op {
+ unknown = -1,
+ eq,
+ ne,
+ lt,
+ gt,
+ le,
+ ge
+};
+
+struct pm_info {
+ int magic;
+ cib_t *cib;
+ unsigned int loglevel;
+ GHashTable *attr_hash;
+ char *uname;
+ char *uuid;
+ char *default_name;
+ char *default_value;
+ gboolean init_attr_done;
+};
+
+struct attr_s {
+ char *name;
+ GList *rules;
+};
+
+struct rule_s {
+ int expr_op;
+ char *expr_value;
+ char *conversion;
+};
+
+#define VALIDATE(arg)
+do {
+ if (!arg || ((struct pm_info *)arg)->magic != MAGIC) {
+ errno = EINVAL;
+ return -1;
+ }
+} while(0)
+
+
+static void
+free_attr_hash(gpointer data)
+{
+ dbg_printf(5, "%s
", __FUNCTION__);
+
+ struct attr_s *attr = data;
+ GList *list = NULL;
+
+ for (list = g_list_first(attr->rules); list; list = g_list_next(list)) {
+ struct rule_s *rule = list->data;
+ crm_free(rule->expr_value);
+ crm_free(rule->conversion);
+ crm_free(rule);
+ }
+ crm_free(attr);
+}
+
+static void
+disconnect_cib(cib_t **cib)
+{
+ dbg_printf(5, "%s
", __FUNCTION__);
+
+ if (*cib) {
+ (*cib)->cmds->signoff(*cib);
+ *cib = NULL;
+ cib = NULL;
+ }
+}
+
+static gboolean
+connect_cib(cib_t **cib, int max_try)
+{
+ enum cib_errors rc = cib_ok;
+ int i;
+
+ dbg_printf(5, "%s
", __FUNCTION__);
+
+ *cib = cib_new();
+ if (!*cib) {
+ syslog(LOG_NOTICE, "cib connection initialization failed
");
+ printf("cib connection initialization failed
");
+ return FALSE;
+ }
+ for (i = 1; i <= max_try; i++) {
+ if (i) sleep(1);
+ dbg_printf(4, "%s: connect to cib attempt: %d
", __FUNCTION__, i);
+ rc = (*cib)->cmds->signon(*cib, crm_system_name, cib_command);
+ if (rc == cib_ok)
+ break;
+ }
+ if (rc != cib_ok) {
+ syslog(LOG_NOTICE, "failed to signon to cib: %s
", cib_error2string(rc));
+ printf("failed to signon to cib: %s
", cib_error2string(rc));
+ disconnect_cib(cib);
+ return FALSE;
+ }
+ dbg_printf(3, "%s: succeed at connect to cib
", __FUNCTION__);
+ return TRUE;
+}
+
+static int
+char2op(const char *str)
+{
+ dbg_printf(5, "%s
", __FUNCTION__);
+
+ int op = unknown;
+
+ if (safe_str_eq(str, "eq")) op = eq;
+ else if (safe_str_eq(str, "ne") || safe_str_eq(str, "neq")) op = ne;
+ else if (safe_str_eq(str, "lt")) op = lt;
+ else if (safe_str_eq(str, "gt")) op = gt;
+ else if (safe_str_eq(str, "le") || safe_str_eq(str, "lte")) op = le;
+ else if (safe_str_eq(str, "ge") || safe_str_eq(str, "gte")) op = ge;
+ return op;
+}
+
+static int
+char2level(const char *str)
+{
+ dbg_printf(5, "%s
", __FUNCTION__);
+
+ if (!str)
+ return 0;
+ if (safe_str_eq(str, "emerg")) return LOG_EMERG;
+ else if (safe_str_eq(str, "alert")) return LOG_ALERT;
+ else if (safe_str_eq(str, "crit")) return LOG_CRIT;
+ else if (safe_str_eq(str, "err") ||
+ safe_str_eq(str, "error")) return LOG_ERR;
+ else if (safe_str_eq(str, "warning") ||
+ safe_str_eq(str, "warn")) return LOG_WARNING;
+ else if (safe_str_eq(str, "notice")) return LOG_NOTICE;
+ else if (safe_str_eq(str, "info")) return LOG_INFO;
+ else if (safe_str_eq(str, "debug")) return LOG_DEBUG;
+ else if (safe_str_eq(str, "debug2")) return LOG_DEBUG + 1;
+ else if (safe_str_eq(str, "debug3")) return LOG_DEBUG + 2;
+ else if (safe_str_eq(str, "debug4")) return LOG_DEBUG + 3;
+ else if (safe_str_eq(str, "debug5")) return LOG_DEBUG + 4;
+ else if (safe_str_eq(str, "debug6")) return LOG_DEBUG + 5;
+ return 0;
+}
+
+static const char *
+convert_attr(GHashTable *attr_hash, const char *name, const char *value)
+{
+ struct attr_s *attr = NULL;
+ GList *list = NULL;
+ char *end_text = NULL;
+ int left, right;
+
+ dbg_printf(5, "%s
", __FUNCTION__);
+
+ attr = g_hash_table_lookup(attr_hash, name);
+ if (!attr) {
+ dbg_printf(2, "%s: There was no conversion rule of %s
",
+ __FUNCTION__, name);
+ return value;
+ }
+
+ for (list = g_list_first(attr->rules); list; list = g_list_next(list)) {
+ struct rule_s *rule = list->data;
+ switch (rule->expr_op) {
+ case eq:
+ dbg_printf(3, "%s: if (cib)%s == %s, respond %s
", __FUNCTION__,
+ value, rule->expr_value, rule->conversion);
+ if (safe_str_eq(value, rule->expr_value))
+ return rule->conversion;
+ break;
+ case ne:
+ dbg_printf(3, "%s: if (cib)%s != %s, respond %s
", __FUNCTION__,
+ value, rule->expr_value, rule->conversion);
+ if (safe_str_neq(value, rule->expr_value))
+ return rule->conversion;
+ break;
+ }
+
+ left = crm_int_helper(value, &end_text);
+ if (errno || end_text[0])
+ continue;
+ dbg_printf(4, "%s: before=%s, after=%d
",
+ __FUNCTION__, value, left);
+ right = crm_int_helper(rule->expr_value, &end_text);
+ if (errno || end_text[0])
+ continue;
+ dbg_printf(4, "%s: before=%s, after=%d
",
+ __FUNCTION__, rule->expr_value, right);
+
+ switch (rule->expr_op) {
+ case lt:
+ dbg_printf(3, "%s: if (cib)%d < %d, respond %s
",
+ __FUNCTION__, left, right, rule->conversion);
+ if (left < right)
+ return rule->conversion;
+ break;
+ case gt:
+ dbg_printf(3, "%s: if (cib)%d > %d, respond %s
",
+ __FUNCTION__, left, right, rule->conversion);
+ if (left > right)
+ return rule->conversion;
+ break;
+ case le:
+ dbg_printf(3, "%s: if (cib)%d <= %d, respond %s
",
+ __FUNCTION__, left, right, rule->conversion);
+ if (left <= right)
+ return rule->conversion;
+ break;
+ case ge:
+ dbg_printf(3, "%s: if (cib)%d >= %d, respond %s
",
+ __FUNCTION__, left, right, rule->conversion);
+ if (left >= right)
+ return rule->conversion;
+ break;
+ }
+ }
+ dbg_printf(2, "%s: attribute %s fulfilled no rule
", __FUNCTION__, value);
+ return value;
+}
+
+static int
+get_attr(struct pm_info *info, const char *name, char **out)
+{
+ char *value = NULL;
+ int rc;
+
+ dbg_printf(5, "%s
", __FUNCTION__);
+
+#ifdef PM_1_0
+ rc = read_attr(info->cib, XML_CIB_TAG_STATUS,
+ info->uuid, NULL, NULL, name, &value, FALSE);
+#else
+ rc = read_attr(info->cib, XML_CIB_TAG_STATUS,
+ info->uuid, NULL, NULL, NULL, name, &value, FALSE);
+#endif
+ if (rc == cib_NOTEXISTS) {
+ crm_free(value);
+ return 1;
+ } else if (rc != cib_ok) {
+ syslog(LOG_NOTICE,
+ "failed to get attribute %s: %s
", name, cib_error2string(rc));
+ printf("failed to get attribute %s: %s
", name, cib_error2string(rc));
+ return -1;
+ }
+
+ if (out) {
+ /* free after use */
+ *out = crm_strdup(convert_attr(info->attr_hash, name, value));
+ }
+ crm_free(value);
+ return 0;
+}
+
+static int
+init_attr(struct pm_info *info, const char *name, const char *value)
+{
+ int i;
+
+ dbg_printf(5, "%s
", __FUNCTION__);
+
+ if (!get_attr(info, name, NULL))
+ return 0;
+
+ if (attrd_lazy_update('U', NULL, name, value, NULL, NULL, 0) == FALSE) {
+ syslog(LOG_NOTICE, "failed in update of the attribute value
");
+ printf("failed in update of the attribute value
");
+ return -1;
+ }
+ for (i = 1; i <= 20; i++) {
+ dbg_printf(4, "%s: waiting..[%d]
", __FUNCTION__, i);
+ sleep(1);
+ if (!get_attr(info, name, NULL))
+ return 0;
+ }
+ return -1;
+}
+
+static int
+check_cib_status(struct pm_info **info, int max_try)
+{
+ struct pm_info *p = *info;
+
+ dbg_printf(5, "%s
", __FUNCTION__);
+
+ if (p->cib) {
+ if (get_attr(p, "dummy", NULL) >= 0)
+ return 0;
+ p->cib->cmds->signoff(p->cib);
+ p->init_attr_done = FALSE;
+ if (p->cib->cmds->signon(p->cib, crm_system_name, cib_command) != cib_ok)
+ return 1;
+ } else {
+ if (connect_cib(&p->cib, max_try) == FALSE)
+ return 1;
+ }
+
+ if (!p->uuid && query_node_uuid(p->cib, p->uname, &p->uuid)) {
+ syslog(LOG_NOTICE, "failed to get node uuid
");
+ printf("failed to get node uuid
");
+ return -1;
+ }
+ if (p->init_attr_done == FALSE && p->default_name && p->default_value) {
+ if (init_attr(p, p->default_name, p->default_value) < 0)
+ return -1;
+ }
+ p->init_attr_done = TRUE;
+ return 0;
+}
+
+static int
+read_config(const char *path, GHashTable **attr_hash)
+{
+ GKeyFile *conf = g_key_file_new();
+ char **groups = NULL;
+ char **keys = NULL;
+ char *value = NULL;
+ char **values = NULL;
+ struct attr_s *attr = NULL;
+ gsize groups_len = 0;
+ gsize keys_len = 0;
+ GError *e = NULL;
+ int i, j;
+
+ dbg_printf(5, "%s
", __FUNCTION__);
+ dbg_printf(3, "%s: config file [%s]
", __FUNCTION__, path);
+
+ *attr_hash = g_hash_table_new_full(
+ g_str_hash, g_str_equal, g_free, free_attr_hash);
+ if (g_key_file_load_from_file(conf, path, G_KEY_FILE_NONE, &e) == FALSE) {
+ syslog(LOG_INFO, "read %s: %s
", path, e->message);
+ printf("read %s: %s
", path, e->message);
+ goto out_fail;
+ }
+
+ groups = g_key_file_get_groups(conf, &groups_len);
+ for (i = 0; i < groups_len; i++) {
+ dbg_printf(4, "%s: group=%s
", __FUNCTION__, groups[i]);
+ keys = g_key_file_get_keys(conf, groups[i], &keys_len, &e);
+ if (!keys) {
+ syslog(LOG_INFO, "group %s: %s
", groups[i], e->message);
+ printf("group %s: %s
", groups[i], e->message);
+ goto out_fail;
+ }
+ crm_malloc0(attr, sizeof(struct attr_s));
+ attr->name = crm_strdup(groups[i]);
+
+ for (j = 0; j < keys_len; j++) {
+ struct rule_s *rule = NULL;
+ crm_malloc0(rule, sizeof(struct rule_s));
+
+ dbg_printf(4, "%s: group=%s, key=%s
",
+ __FUNCTION__, groups[i], keys[j]);
+ value = g_key_file_get_value(conf, groups[i], keys[j], &e);
+ if (!value) {
+ syslog(LOG_INFO, "group %s, key %s: %s
",
+ groups[i], keys[j], e->message);
+ printf("group %s, key %s: %s
",
+ groups[i], keys[j], e->message);
+ crm_free(rule);
+ goto out_fail;
+ }
+ values = g_strsplit(value, " ", 2);
+ if (g_strv_length(values) < 2) {
+ syslog(LOG_INFO,
+ "unjust rule %s, ignore section %s - %s
",
+ value, groups[i], keys[j]);
+ printf("unjust rule %s, ignore section %s - %s
",
+ value, groups[i], keys[j]);
+ crm_free(rule);
+ goto free;
+ }
+ rule->expr_op = char2op(values[0]);
+ if (rule->expr_op == unknown) {
+ syslog(LOG_INFO,
+ "unjust operation %s, ignore section %s - %s
",
+ values[0], groups[i], keys[j]);
+ printf("unjust operation %s, ignore section %s - %s
",
+ values[0], groups[i], keys[j]);
+ crm_free(rule);
+ goto free;
+ }
+ dbg_printf(3, "%s: group=%s, key=%s, value=%s
",
+ __FUNCTION__, groups[i], keys[j], value);
+ rule->expr_value = crm_strdup(values[1]);
+ rule->conversion = crm_strdup(keys[j]);
+ attr->rules = g_list_append(attr->rules, rule);
+free:
+ g_strfreev(values);
+ crm_free(value);
+ }
+ g_hash_table_insert(*attr_hash, attr->name, attr);
+ g_strfreev(keys);
+ }
+ g_strfreev(groups);
+ g_key_file_free(conf);
+ return 0;
+out_fail:
+ g_key_file_free(conf);
+ if (e) g_error_free(e);
+ if (groups) g_strfreev(groups);
+ if (keys) g_strfreev(keys);
+ if (attr) crm_free(attr);
+ return -1;
+}
+
+static void
+reset_lib_log(unsigned int level)
+{
+ dbg_printf(5, "%s
", __FUNCTION__);
+
+ cl_log_set_entity(BACKEND_NAME);
+ set_crm_log_level(level);
+}
+
+
+static int
+pm_hoststatus(hoststatus_callback callback, int fd, const char *req, void *priv)
+{
+ struct pm_info *info = (struct pm_info *)priv;
+ char **req_arr;
+ char *value = NULL;
+ int i;
+
+ dbg_printf(5, "%s
", __FUNCTION__);
+
+ VALIDATE(info);
+ reset_lib_log(info->loglevel);
+
+ if (check_cib_status(&info, 20))
+ return 1;
+
+ req_arr = g_strsplit(req, " ", 0);
+ for (i = 0; i < g_strv_length(req_arr); i++) {
+ dbg_printf(2, "%s: request-%d: %s
", __FUNCTION__, i+1, req_arr[i]);
+ if (!strlen(req_arr[i])) {
+ dbg_printf(2, "%s: ignore the empty token
", __FUNCTION__);
+ continue;
+ }
+ get_attr(info, req_arr[i], &value);
+ callback(req_arr[i], value ? value : "", fd);
+ crm_free(value);
+ }
+ g_strfreev(req_arr);
+ return 0;
+}
+
+
+static int
+pm_init(backend_context_t *c, config_object_t *conf)
+{
+ struct pm_info *info = NULL;
+ int level = 0;
+ char value[256];
+ char key[32];
+ struct utsname name;
+
+ dbg_printf(5, "%s
", __FUNCTION__);
+
+#ifdef _MODULE
+ if (sc_get(conf, "fence_virtd/@debug", value, sizeof(value)) == 0)
+ dset(atoi(value));
+#endif
+ sprintf(key, "general_backends/%s/@pmlib_loglevel", BACKEND_NAME);
+ if (sc_get(conf, key, value, sizeof(value)) == 0) {
+ level = char2level(value);
+ crm_log_init(BACKEND_NAME, level, FALSE, FALSE, 0, NULL);
+ cl_log_enable_stdout(TRUE);
+ }
+
+ info = malloc(sizeof(*info));
+ if (!info)
+ return -1;
+
+ memset(info, 0, sizeof(*info));
+ info->magic = MAGIC;
+ info->loglevel = level;
+ info->init_attr_done = FALSE;
+
+ if (uname(&name) < 0) {
+ perror("uname");
+ return -1;
+ }
+ info->uname = crm_strdup(name.nodename);
+
+ sprintf(key, "general_backends/%s/@config", BACKEND_NAME);
+ if (sc_get(conf, key, value, sizeof(value)) != 0)
+ strcpy(value, CONFIG_FILE);
+ if (read_config(value, &info->attr_hash)) {
+ syslog(LOG_NOTICE, "failed to read config file
");
+ printf("failed to read config file
");
+ return -1;
+ }
+
+ sprintf(key, "general_backends/%s/@default_attr_name", BACKEND_NAME);
+ if (sc_get(conf, key, value, sizeof(value)) == 0)
+ info->default_name = crm_strdup(value);
+ sprintf(key, "general_backends/%s/@default_attr_value", BACKEND_NAME);
+ if (sc_get(conf, key, value, sizeof(value)) == 0)
+ info->default_value = crm_strdup(value);
+ if ((info->default_name && !info->default_value) ||
+ (!info->default_name && info->default_value)) {
+ syslog(LOG_NOTICE,
+ "Bad setting: %s/default_attr_name or %s/default_attr_value
",
+ BACKEND_NAME, BACKEND_NAME);
+ printf("Bad setting: %s/default_attr_name or %s/default_attr_value
",
+ BACKEND_NAME, BACKEND_NAME);
+ return -1;
+ } else if (info->default_name && info->default_value) {
+ if (check_cib_status(&info, 1) < 0)
+ return -1;
+ }
+ *c = (void *)info;
+ return 0;
+}
+
+
+static int
+pm_shutdown(backend_context_t c)
+{
+ struct pm_info *info = (struct pm_info *)c;
+
+ dbg_printf(5, "%s
", __FUNCTION__);
+
+ VALIDATE(info);
+ reset_lib_log(info->loglevel);
+
+ disconnect_cib(&info->cib);
+
+ if (info->attr_hash)
+ g_hash_table_destroy(info->attr_hash);
+ if (info->init_attr_done == TRUE && info->default_name && info->default_value) {
+ if (attrd_lazy_update('D', NULL,
+ info->default_name, NULL, NULL, NULL, 0) == FALSE) {
+ syslog(LOG_NOTICE, "failed in delete of the attribute value
");
+ printf("failed in delete of the attribute value
");
+ }
+ }
+ if (info->default_name) crm_free(info->default_name);
+ if (info->default_value) crm_free(info->default_value);
+ if (info->uname) crm_free(info->uname);
+ if (info->uuid) crm_free(info->uuid);
+ free(info);
+ return 0;
+}
+
+
+static fence_callbacks_t pm_callbacks = {
+ .hoststatus = pm_hoststatus
+};
+
+static backend_plugin_t pm_plugin = {
+ .name = BACKEND_NAME,
+ .version = VERSION,
+ .callbacks = &pm_callbacks,
+ .init = pm_init,
+ .cleanup = pm_shutdown,
+};
+
+
+#ifdef _MODULE
+double
+BACKEND_VER_SYM(void)
+{
+ return PLUGIN_VERSION_BACKEND;
+}
+
+const backend_plugin_t *
+BACKEND_INFO_SYM(void)
+{
+ return &pm_plugin;
+}
+#else
+static void __attribute__((constructor))
+pm_register_plugin(void)
+{
+ plugin_reg_backend(&pm_plugin);
+}
+#endif
diff -urN fence-virt-200eab4/server/serial.c mod/server/serial.c
--- fence-virt-200eab4/server/serial.c 2011-09-20 10:17:15.078381411 +0900
+++ mod/server/serial.c 2011-09-20 10:40:23.131631616 +0900
@@ -71,6 +71,8 @@
uint64_t magic;
const fence_callbacks_t *cb;
void *priv;
+ const fence_callbacks_t *cb2;
+ void *priv2;
char *uri;
char *path;
history_info_t *history;
@@ -171,6 +173,62 @@


static int
+serial_hoststatus(const char *req, const char *res, int fd)
+{
+ host_status_t hstatus;
+ struct timeval tv;
+ int ret;
+
+ strncpy((char *)hstatus.req, req, sizeof(hstatus.req));
+ strncpy((char *)hstatus.res, res, sizeof(hstatus.res));
+ tv.tv_sec = 1;
+ tv.tv_usec = 0;
+
+ dbg_printf(1, "%s: request [%s], response [%s]
",
+ __FUNCTION__, (char*)hstatus.req, (char*)hstatus.res);
+ ret = _write_retry(fd, &hstatus, sizeof(hstatus), &tv);
+ if (ret == sizeof(hstatus))
+ return 0;
+ return 1;
+}
+
+
+static int
+serial_hoststatus_begin(int fd)
+{
+ struct timeval tv;
+ serial_resp_t resp;
+
+ resp.magic = SERIAL_MAGIC;
+ resp.response = RESP_HOSTSTATUS;
+
+ tv.tv_sec = 1;
+ tv.tv_usec = 0;
+ return _write_retry(fd, &resp, sizeof(resp), &tv);
+}
+
+
+static int
+serial_hoststatus_end(int fd)
+{
+ host_status_t hstatus;
+ struct timeval tv;
+ int ret;
+
+ //printf("Sending terminator packet
");
+
+ memset(&hstatus, 0, sizeof(hstatus));
+
+ tv.tv_sec = 1;
+ tv.tv_usec = 0;
+ ret = _write_retry(fd, &hstatus, sizeof(hstatus), &tv);
+ if (ret == sizeof(hstatus))
+ return 0;
+ return 1;
+}
+
+
+static int
do_fence_request(int fd, const char *src, serial_req_t *req, serial_info *info)
{
char response = RESP_FAIL;
@@ -181,7 +239,10 @@

switch(req->request) {
case FENCE_NULL:
- response = info->cb->null((char *)req->domain, info->priv);
+ if (info->cb)
+ response = info->cb->null((char *)req->domain, info->priv);
+ else
+ response = RESP_NOTSUPPORT;
break;
case FENCE_ON:
if (map_check(info->maps, src,
@@ -189,8 +250,12 @@
response = RESP_PERM;
break;
}
- response = info->cb->on((char *)req->domain, src,
+
+ if (info->cb)
+ response = info->cb->on((char *)req->domain, src,
req->seqno, info->priv);
+ else
+ response = RESP_NOTSUPPORT;
break;
case FENCE_OFF:
if (map_check(info->maps, src,
@@ -198,8 +263,12 @@
response = RESP_PERM;
break;
}
- response = info->cb->off((char *)req->domain, src,
+
+ if (info->cb)
+ response = info->cb->off((char *)req->domain, src,
req->seqno, info->priv);
+ else
+ response = RESP_NOTSUPPORT;
break;
case FENCE_REBOOT:
if (map_check(info->maps, src,
@@ -207,29 +276,48 @@
response = RESP_PERM;
break;
}
- response = info->cb->reboot((char *)req->domain, src,
+
+ if (info->cb)
+ response = info->cb->reboot((char *)req->domain, src,
req->seqno, info->priv);
+ else
+ response = RESP_NOTSUPPORT;
break;
case FENCE_STATUS:
- if (map_check(info->maps, src,
- (const char *)req->domain) == 0) {
- response = RESP_PERM;
- break;
- }
- response = info->cb->status((char *)req->domain, info->priv);
+ if (info->cb)
+ response = info->cb->status((char *)req->domain, info->priv);
+ else
+ response = RESP_NOTSUPPORT;
break;
case FENCE_DEVSTATUS:
- response = info->cb->devstatus(info->priv);
+ if (info->cb)
+ response = info->cb->devstatus(info->priv);
+ else
+ response = RESP_NOTSUPPORT;
break;
case FENCE_HOSTLIST:
arg.map = info->maps;
arg.src = src;
arg.fd = fd;

- serial_hostlist_begin(arg.fd);
- response = info->cb->hostlist(serial_hostlist, &arg,
+ if (info->cb) {
+ serial_hostlist_begin(arg.fd);
+ response = info->cb->hostlist(serial_hostlist, &arg,
info->priv);
- serial_hostlist_end(arg.fd);
+ serial_hostlist_end(arg.fd);
+ } else {
+ response = RESP_NOTSUPPORT;
+ }
+ break;
+ case MONITOR_HOSTSTATUS:
+ if (info->cb2) {
+ serial_hoststatus_begin(fd);
+ response = info->cb2->hoststatus(serial_hoststatus, fd,
+ (const char *)req->query, info->priv2);
+ serial_hoststatus_end(fd);
+ } else {
+ response = RESP_NOTSUPPORT;
+ }
break;
}

@@ -364,8 +452,8 @@


static int
-serial_init(listener_context_t *c, const fence_callbacks_t *cb,
- config_object_t *config, map_object_t *map, void *priv)
+serial_init(listener_context_t *c, const fence_callbacks_t *cb, const fence_callbacks_t *cb2,
+ config_object_t *config, map_object_t *map, void *priv, void *priv2)
{
serial_info *info;
int ret;
@@ -377,6 +465,8 @@

info->priv = priv;
info->cb = cb;
+ info->priv2 = priv2;
+ info->cb2 = cb2;

ret = serial_config(config, info);
if (ret < 0) {
#!/bin/sh
#
#
# vm-client OCF RA. refer for arbitrary attribute information for
# fence-virt.
#
# Copyright (c) 2010 NIPPON TELEGRAPH AND TELEPHONE CORPORATION
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of version 2 of the GNU General Public License as
# published by the Free Software Foundation.
#
# This program is distributed in the hope that it would be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#
# Further, this software is distributed without any warranty that it is
# free of the rightful claim of any third person regarding infringement
# or the like. Any license provided herein, whether implied or
# otherwise, applies only to this software file. Patent licenses, if
# any, provided herein do not apply to combinations of this program with
# other software, or any other product whatsoever.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write the Free Software Foundation,
# Inc., 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
#

################################################## #####################
# Initialization:

. ${OCF_ROOT}/resource.d/heartbeat/.ocf-shellfuncs

################################################## #####################

meta_data() {
cat <<END
<?xml version="1.0"?>
<!DOCTYPE resource-agent SYSTEM "ra-api-1.dtd">
<resource-agent name="vm-client" version="1.0">
<version>1.0</version>

<longdesc lang="en">
This Resource Agent is fence-virt and a thing to communicate in virtual environment.
</longdesc>
<shortdesc lang="en">vm-client resource agent</shortdesc>

<parameters>

<parameter name="fence_virt_params" unique="0" require="1">
<longdesc lang="en">
Specify the parameters of the fencing agent command (fence_virt).
The following options need not be specified.
-o : action.
-t : timeout.
For example: "fence_virt -D /dev/ttyS1"
</longdesc>
<shortdesc lang="en">parameters of fence_virt command</shortdesc>
<content type="string" default="" />
</parameter>

<parameter name="attribute_list" unique="1" require="1">
<longdesc lang="en">
The list of the attribute to refer to host.
</longdesc>
<shortdesc lang="en">attribute list</shortdesc>
<content type="string" default="" />
</parameter>

<parameter name="state" unique="1">
<longdesc lang="en">
Location to store the resource state in.
</longdesc>
<shortdesc lang="en">State file</shortdesc>
<content type="string" default="${HA_VARRUN}/vm-client-${OCF_RESOURCE_INSTANCE}.state" />
</parameter>

<parameter name="debug" unique="0">
<longdesc lang="en">
Enables to use default ${ATTRD_UPDATER} verbose logging on every call.
</longdesc>
<shortdesc lang="en">Verbose logging</shortdesc>
<content type="string" default="false"/>
</parameter>

</parameters>

<actions>
<action name="start" timeout="90" />
<action name="stop" timeout="100" />
<action name="monitor" timeout="20" interval="10" depth="0" start-delay="0" />
<action name="reload" timeout="90" />
<action name="meta-data" timeout="5" />
<action name="validate-all" timeout="30" />
</actions>
</resource-agent>
END
}

################################################## #####################

vmclient_usage() {
cat <<END
usage: $0 {start|stop|monitor|validate-all|meta-data}

Expects to have a fully populated OCF RA-compliant environment set.
END
}

attrd_send_update() {
echo ${1} | grep "=" 2>&1 >/dev/null
if [ $? -ne 0 ]; then
return 0
fi

name=`echo ${1} | sed 's/=.*//g'`
value=`echo ${1} | sed 's/.*=//g'`

if [ ${value}x = "x" ]; then
${ATTRD_UPDATER} -D -n ${name} ${logging_options}
return 0
fi

${ATTRD_UPDATER} -n ${name} -U ${value} ${logging_options}
if [ $? -ne 0 ]; then
ocf_log err "attrd failed to update. (attr_name=$name)"
exit $OCF_ERR_GENERIC
fi
}

communicate_fence_virtd() {
while true; do
ocf_log debug "Request: ${FENCING_AGENT} -t3600 ${params} -ohost_status -Q"${attrs}""
res=`${FENCING_AGENT} -t3600 ${params} -ohost_status -Q"${attrs}"`
rc=$?
if [ $rc -eq 0 ]; then
ocf_log debug "Result: $res"
IFS=","
for attr in ${res}; do
attrd_send_update "${attr}"
done
return $OCF_SUCCESS
elif [ $rc -eq 1 -o $rc -eq 2 -o $rc -eq 3 ]; then
ocf_log err "request failed."
return $OCF_ERR_GENERIC
else
ocf_log info "request failed."
sleep 3
continue
fi
done
}

vmclient_start() {
vmclient_monitor
if [ $? -eq $OCF_SUCCESS ]; then
return $OCF_SUCCESS
fi

touch ${OCF_RESKEY_state}
vmclient_monitor
}

vmclient_stop() {
rm -f ${OCF_RESKEY_state}
for name in ${attrs}; do
${ATTRD_UPDATER} -D -n ${name} ${logging_options}
if [ $? -ne 0 ]; then
ocf_log err "attrd failed to delete. (attr_name=$name)"
return $OCF_ERR_GENERIC
fi
done
return $OCF_SUCCESS
}

vmclient_monitor() {
if [ -f ${OCF_RESKEY_state} ]; then
communicate_fence_virtd
if [ $? -ne $OCF_SUCCESS ]; then
ocf_log err "failed in the reception from fence_virtd."
return $OCF_ERR_GENERIC
fi
return $OCF_SUCCESS
fi
return $OCF_NOT_RUNNING
}

vmclient_validate() {
# Is the state directory writable?
state_dir=`dirname "$OCF_RESKEY_state"`
touch "$state_dir/$$"
if [ $? != 0 ]; then
ocf_log err "Invalid location for 'state': $state_dir is not writable"
return $OCF_ERR_ARGS
fi
rm "$state_dir/$$"

# Pidfile better be an absolute path
case $OCF_RESKEY_state in
/*) ;;
*) ocf_log warn "You should use an absolute path for state file not: $OCF_RESKEY_state" ;;
esac

# Check the attribute list
if [ "x" = "x$attrs" ]; then
ocf_log err "Empty attribute_list."
exit $OCF_ERR_CONFIGURED
fi

check_binary ${FENCING_AGENT}
check_binary ${ATTRD_UPDATER}
return $OCF_SUCCESS
}

: ${OCF_RESKEY_CRM_meta_interval=0}
: ${OCF_RESKEY_CRM_meta_globally_unique:="true"}
: ${OCF_RESKEY_debug:="false"}

params=`echo ${OCF_RESKEY_fence_virt_params}`
attrs=`echo ${OCF_RESKEY_attribute_list} | tr -s " "`

if [ "x$OCF_RESKEY_state" = "x" ]; then
if [ ${OCF_RESKEY_CRM_meta_globally_unique} = "false" ]; then
state="${HA_VARRUN}/vm-client-${OCF_RESOURCE_INSTANCE}.state"
# Strip off the trailing clone marker
OCF_RESKEY_state=`echo $state | sed s/:[0-9][0-9]*.state/.state/`
else
OCF_RESKEY_state="${HA_VARRUN}/vm-client-${OCF_RESOURCE_INSTANCE}.state"
fi
fi

logging_options='-q'
if ocf_is_true ${OCF_RESKEY_debug} ; then
logging_options='
fi

FENCING_AGENT="fence_virt"
ATTRD_UPDATER="attrd_updater"

case $__OCF_ACTION in
meta-data) meta_data
exit $OCF_SUCCESS
;;
start) vmclient_start;;
stop) vmclient_stop;;
monitor) vmclient_monitor;;
reload) vmclient_start;;
validate-all) vmclient_validate;;
usage|help) vmclient_usage
exit $OCF_SUCCESS
;;
*) vmclient_usage
exit $OCF_ERR_UNIMPLEMENTED
;;
esac
exit $?
fence_virtd {
listener = "serial";
backend = "libvirt";
module_path = "/usr/lib64/fence-virt";
general_backend = "pm-monitor";
}

listeners {
serial {
mode = "serial";
path = "/var/lib/libvirt/qemu";
}
}

backends {
libvirt {
uri = "qemu:///system";
}
}

general_backends {
pm-monitor {
config = "/etc/pm-monitor.conf";
# pmlib_loglevel = "debug2";
default_attr_name = "operator_check_status";
default_attr_value = "50";
}
}
[default_ping_set]
red = eq 0
yellow = lt 100
green = eq 100
property no-quorum-policy="freeze"
stonith-enabled="false"
startup-fencing="false"

rsc_defaults resource-stickiness="INFINITY"
migration-threshold="5"

primitive prmGuest-a1 ocf:extra:VirtualDomain
meta allow-migrate="true"
params config="/etc/libvirt/qemu/srv-a1.xml" hypervisor="qemu:///system" migration_transport="ssh"
op start timeout="120s" on-fail="restart"
op monitor interval="10s" timeout="30s" on-fail="restart"
op stop timeout="120s" on-fail="block"
op migrate_to interval="0s" timeout="120s" on-fail="block"
op migrate_from interval="0s" timeout="120s" on-fail="restart"

primitive prmGuest-a2 ocf:extra:VirtualDomain
meta allow-migrate="true"
params config="/etc/libvirt/qemu/srv-a2.xml" hypervisor="qemu:///system" migration_transport="ssh"
op start timeout="120s" on-fail="restart"
op monitor interval="10s" timeout="30s" on-fail="restart"
op stop timeout="120s" on-fail="block"
op migrate_to interval="0s" timeout="120s" on-fail="block"
op migrate_from interval="0s" timeout="120s" on-fail="restart"

clone clnPingd
prmPingd

primitive prmPingd ocfacemakering
params
name="default_ping_set"
host_list="192.168.201.254"
multiplier="100"
dampen="5"
meta
migration-threshold="10"
op start timeout="90s" on-fail="restart"
op monitor interval="10s" timeout="60s" on-fail="restart"
op stop timeout="100s" on-fail="ignore"
property no-quorum-policy="ignore"
stonith-enabled="false"
startup-fencing="false"

rsc_defaults resource-stickiness="INFINITY"
migration-threshold="1"

primitive prmDummy ocfacemakerummy
op start timeout="90s" on-fail="restart"
op monitor interval="10s" timeout="20s" on-fail="restart"
op stop timeout="100s" on-fail="fence"

clone clnVmClient
prmVmClient
primitive prmVmClient ocf:extra:vm-client
params
fence_virt_params="-D/dev/ttyS1"
attribute_list="default_ping_set operator_check_status"
op start timeout="90s" on-fail="restart"
op monitor interval="10s" timeout="60s" on-fail="restart"
op stop timeout="100s" on-fail="block"
[root@x3650g ~]# crm_mon -rfA1
============
Last updated: Tue Oct 4 14:17:10 2011
Last change: Tue Oct 4 14:11:54 2011 via crmd on x3650g
Stack: Heartbeat
Current DC: x3650g (32611d84-9cb5-410e-af1d-7b2c1ad2e443) - partition with quorum
Version: 1.1.6-1.el6-9971ebba4494012a93c03b40a2c58ec0eb60f50c
2 Nodes configured, unknown expected votes
2 Resources configured.
============

Online: [ x3650f x3650g ]

Full list of resources:

Clone Set: clnPingd [prmPingd]
Started: [ x3650f x3650g ]

Node Attributes:
* Node x3650f:
+ default_ping_set : 100
+ operator_check_status : 50
* Node x3650g:
+ default_ping_set : 100
+ operator_check_status : 50

Migration summary:
* Node x3650g:
* Node x3650f:
[root@x3650g ~]#[root@srv-a2 ~]# crm_mon -rfA1
============
Last updated: Tue Oct 4 14:17:36 2011
Last change: Tue Oct 4 14:15:29 2011 via cibadmin on srv-a1
Stack: Heartbeat
Current DC: srv-a2 (68057c1e-6a5e-402b-be84-c56b98d5c985) - partition with quorum
Version: 1.1.6-1.el6-9971ebba4494012a93c03b40a2c58ec0eb60f50c
2 Nodes configured, unknown expected votes
3 Resources configured.
============

Online: [ srv-a2 srv-a1 ]

Full list of resources:

prmDummy (ocf:acemakerummy): Started srv-a1
Clone Set: clnVmClient [prmVmClient]
Started: [ srv-a2 srv-a1 ]

Node Attributes:
* Node srv-a2:
+ default_ping_set : green
+ operator_check_status : GREEN
* Node srv-a1:
+ default_ping_set : green
+ operator_check_status : GREEN

Migration summary:
* Node srv-a1:
* Node srv-a2:
[root@srv-a2 ~]#
 
Old 10-26-2011, 01:35 AM
Lon Hohberger
 
Default backend plugin for monitoring a host's status

On 10/05/2011 04:56 AM, Kazunori INOUE wrote:
> Hi all,
>
> I think that the communication function of fence-virt is flexible,
> so I want to use it more effectively.
>
> Therefore I made backend plugin for a guest to get the host's status
> using the communication facility of fence-virt,
> and I changed to allow specifying one more backend (for fencing, and
> replying the host's status).
>
> I created the backend "pm-monitor" which has met the following
> configurations / requirements.
> - Both hosts and VMs, cluster (Pacemaker) have been configured.
>
> Here's an overview of function. Please refer to attached 'overview.png'.
> (*) pingd resource notifies the status of connection with a specific
> host to pacemaker, and pacemaker manages the result.
> (1) resource (vm-client) which requires the host's status is executed.
> (2) vm-client requests 'host_status (result of pingd)' to the host
> with fence_virt.
> (3) use the serial listener,
> (4) fence_virtd (pm-monitor backend) gets the 'result of pingd' from
> pacemaker and answers it after conversion.
> - the conversion rule is set in /etc/pm-monitor.conf

Originally, the devstatus callback was supposed to be what provided the
answer to the question "is my fencing [device|host] operational?" - but
your patch is more than that, it looks like a generalized way to do
arbitrary request/response pacemaker resource monitoring from within VMs.

That kind of monitoring is interesting.

> Here's a description of the attached files.
> * add_general_backend.patch
> - add the server/pm-fence.c
> - change the configure.in and server/Makefile.in

I think I understand how it operates; your explanation is very detailed.

* I don't quite understand all of the benefits. Presumably, one uses
the serial configuration to prevent guests/hosts from sharing the same
networks - i.e., it's designed for environments where guests and hosts
are not allowed to talk over the network to each other. So, presumably,
we're using this to indirectly monitor an IP on the host network, using
fence-virt as a bridge. What I don't understand is the benefit to the
virtual machine cluster in doing this.

* Do you see other uses besides monitoring pingd sets?

* It may be that this new project may be a better "backbone" for this
sort of general request/response than fence-virt; it's a much more
general communications medium for guest<->host communication:

http://git.fedorahosted.org/git/?p=vios-proxy.git

(although it is written in C++ ).


Notes about the patch itself:

* no support for multicast listener (this looks intentional -
is it?)

* this will break on-wire compatibility; we need to be very careful
here.

* some duplicate build system changes with the other
patch (not a big deal)

* [not directly related to your patch] the vmchannel/serial support
in fence-virt probably should be replaced with more recent
technology in libvirt

-- Lon
 

Thread Tools




All times are GMT. The time now is 06:59 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org